georg's bloghttp://www.wassen.net/2014-11-13T12:24:00+01:00Articles2014-11-13T12:24:00+01:00Georg Wassentag:www.wassen.net,2014-11-13:articles.html<p>Selection of texts from various topics I suggest to read.</p>
<p><a href="http://michaelrbernste.in/2014/10/21/should-i-read-papers.html">Should I read papers?</a>
Spoiler: yes.</p>Vim Regular Expressions2014-09-30T08:53:00+02:00Georg Wassentag:www.wassen.net,2014-09-30:vim-regular-expressions.html<p>This page contains some really cool regular expressions for Vim.
I'll extend them to future reference.</p>
<h1 id="search-multiple-lines">Search multiple lines</h1>
<p>By default, a regex checks line by line. Some suggest to include line wraps <code>\n\r</code>
explicitly (but I've not confirmed if that works).
Vim supports the construct <code>\_.</code> to match any character including line breaks.
The following Regex:</p>
<div class="highlight"><pre>/@incollection\_.\{-}\(\(booktitle\)\|\(crossref\)\)
</pre></div>
<p>searches a BIB file for all <code>@incollection</code> entries and highlights
up to the following <code>booktitle</code> or <code>crossref</code>.
The `{-} ensures a <em>non-greedy</em> behavior, i.o.w. it only goes unto
the <em>shortest</em> possible match (see below).</p>
<p><a href="http://stackoverflow.com/questions/784176/multi-line-regex-support-in-vim">Source</a></p>
<h1 id="find-the-shortest-possible-match">Find the shortest possible match</h1>
<p>Commonly, a regex will match as much characters as possible.
Not only in multi-line matches (as shown above), but sometimes
also in other situations,
a <em>non-greedy</em> behavior is wanted.</p>
<p>A simple approach to match the <em>next</em> character is to exclude that
from the pattern between:</p>
<div class="highlight"><pre>/<[^>]*>
</pre></div>
<p>But if the regex gets more complex, the special multi token <code>\{-}</code> can
be used. It will match the shortest possible pattern.</p>
<p>See <code>:help non-greedy</code> and [Source](multi token <code>\{-}</code> can
be used. It will match the shortest possible pattern.</p>
<p>See <code>:help non-greedy</code> and <a href="http://stackoverflow.com/questions/1305853/how-can-i-make-my-match-non-greedy-in-vim">Source</a></p>
<h1 id="search-for-lines-not-containing-certain-words">Search for lines NOT containing certain words</h1>
<div class="highlight"><pre>/@\(\(article\)\@!\&\(inproceedings\)\@!\&.*\){
</pre></div>
<p>I've a BibLatex file and want to check all entries, that are <strong>not</strong>
of the classes <code>@article</code> and <code>@inproceedings</code> (all entry types
are spelled in lower case, so that I don't have to deal with that).</p>
<p>If the file contains:</p>
<div class="highlight"><pre>01 @article{a,
02 title={Demo article},
03 }
04 @report{b,
05 title={Demo report},
06 }
07 @inproceedings{c,
08 title={Demo presentation},
09 }
10 @misc{d,
11 title={Demo web reference},
12 }
</pre></div>
<p>This search pattern would find the lines containing an <code>@</code> and not article
or inproceedings, but something and than an opening curly brace <code>{</code>.
This matches the lines 4 and 10.</p>Highlight output2014-07-29T11:55:00+02:00Georg Wassentag:www.wassen.net,2014-07-29:highlight-output.html<p>A previous <a href="http://www.wassen.net/colorful-prompts.html">posting</a> explained how to use colors on the bash prompt.
Here is a convenient filter function to highlight words (or regex's) in files or in the output
of commands.</p>
<div class="highlight"><pre><span class="c"># highlight words in output.</span>
<span class="c"># use: cat /etc/passwd | hl root user</span>
<span class="c"># from: http://chneukirchen.org/blog/archive/2013/07/summer-of-scripts-hl.html </span>
<span class="c"># zsh: -e${^*} expands $*="a b c" to -ea -eb -ec</span>
<span class="c"># ported to bash by me using xargs</span>
<span class="k">function</span> hl <span class="o">()</span> <span class="o">{</span>
<span class="k">if</span> <span class="o">[[</span> <span class="nv">$1</span> <span class="o">=</span> <span class="s1">'-i'</span> <span class="o">]]</span><span class="p">;</span> <span class="k">then</span>
<span class="nv">ARGS</span><span class="o">=</span><span class="s1">'--ignore-case'</span>
<span class="nb">shift</span>
<span class="nb"> </span><span class="k">fi</span>
egrep <span class="nv">$ARGS</span> --color<span class="o">=</span>always -e <span class="s1">''</span> <span class="k">$(</span><span class="nb">echo</span> <span class="nv">$*</span> <span class="p">|</span> xargs -n1 <span class="nb">printf</span> <span class="s2">"-e%s "</span><span class="k">)</span><span class="p">;</span>
<span class="o">}</span>
</pre></div>
<p>I have this in my <code>.bash_profile</code>.</p>
<p>The function first checks, if the first parameter is <code>-i</code> to activate an according parameter to <code>egrep</code>.
(If that param was found, <code>shift</code> removes it.)
The main work is done by <code>egrep</code>.
The parameter <code>--color=always</code> activates output coloring
even if it is used in a pipe (or the output is written to a file).
This is required to get colors forwarded to <code>less</code> or further processing.</p>
<p>In the original by <a href="http://chneukirchen.org/blog/archive/2013/07/summer-of-scripts-hl.html">Ch Neukirchen</a>,
the <code>zsh</code> can magically prepend parameters with a string.
This was ported to <code>bash</code> by using a combination of <code>xargs</code> and <code>printf</code>.
Effectively, <code>egrep</code> gets multiple expressions:
The empty string results in all lines being printed (not only the lines with a matching).
But since the expression is empty, it matches but does not color anything.
Then, the arguments of the <code>hl</code> function are listed,
each one prepended with <code>-e</code>.</p>
<div class="highlight"><pre>hl root user
</pre></div>
<p>becomes</p>
<div class="highlight"><pre>egrep --color=always -e '' -e root -e user
</pre></div>
<h1 id="use">Use</h1>
<p>As shown in the comments, this filter can be included in command pipes in the interactive command line:</p>
<div class="highlight"><pre><span class="nv">$ </span>cat /etc/passwd <span class="p">|</span> hl root user
</pre></div>
<p>This will highlight the words 'root' and 'user' in the output.
To highlight an entire line, use a regex:</p>
<div class="highlight"><pre><span class="nv">$ </span>cat /etc/passwd <span class="p">|</span> hl <span class="s1">'.*root.*'</span>
</pre></div>
<h1 id="change-color">Change color</h1>
<p>Multiple instances of <code>hl</code> can be piped together.
Since the coloring is done by <code>egrep</code>, that utility can be instructed to use different colors.
In a Makefile for a LaTeX project, I have included the following:</p>
<div class="highlight"><pre><span class="nf">check </span><span class="o">:</span> <span class="k">$(</span><span class="nv">MAIN</span><span class="k">)</span>.<span class="n">pdf</span>
cat <span class="k">$(</span>MAIN<span class="k">)</span>.log <span class="se">\</span>
<span class="p">|</span> <span class="nv">GREP_COLORS</span><span class="o">=</span><span class="s1">'mt=01;33'</span> egrep --color<span class="o">=</span>always --ignore-case -e <span class="s1">''</span><span class="s2">""</span><span class="s1">''</span> -e<span class="s1">'^.*float specifier changed to.*$$'</span> <span class="se">\</span>
<span class="p">|</span> <span class="nv">GREP_COLORS</span><span class="o">=</span><span class="s1">'mt=01;34'</span> egrep --color<span class="o">=</span>always --ignore-case -e <span class="s1">''</span><span class="s2">""</span><span class="s1">''</span> -e<span class="s1">'^.*underful.*$$'</span> <span class="se">\</span>
<span class="p">|</span> <span class="nv">GREP_COLORS</span><span class="o">=</span><span class="s1">'mt=01;32'</span> egrep --color<span class="o">=</span>always --ignore-case -e <span class="s1">''</span><span class="s2">""</span><span class="s1">''</span> -e<span class="s1">'^.*overful.*$$'</span> <span class="se">\</span>
<span class="p">|</span> <span class="nv">GREP_COLORS</span><span class="o">=</span><span class="s1">'mt=01;31'</span> egrep --color<span class="o">=</span>always --ignore-case -e <span class="s1">''</span><span class="s2">""</span><span class="s1">''</span> -e<span class="s1">'^.*warning.*$$'</span> <span class="se">\</span>
<span class="p">|</span> less
</pre></div>
<p>The environment variable <code>GREP_COLORS</code> is set differently for the various instances of egrep.
The cluster of quote signs <code>''""''</code> is required for the Makefile processing so that the same <code>''</code>
as in the function above remains active (the others are eaten by make and its subshell).</p>
<p>With <code>--color=always</code>, the matching lines are colored and forwarded to the next filter.
Finally, a colorful output is presented in <code>less</code>.</p>
<p>The color codes are the same as in the first table of <a href="http://www.wassen.net/colorful-prompts.html">the color introduction</a>.</p>
<h1 id="references">References</h1>
<ul>
<li><a href="http://chneukirchen.org/blog/archive/2013/07/summer-of-scripts-hl.html">Ch Neukirchen Summer of Scripts</a></li>
<li><a href="http://blog.amit-agarwal.co.in/2011/03/06/grep_colors-change-the-colors-in-the-grep-output/">Grep colors</a></li>
</ul>YAML with Python2014-05-15T10:13:00+02:00Georg Wassentag:www.wassen.net,2014-05-15:yaml-with-python.html<h1 id="yaml">YAML</h1>
<p><a href="http://www.yaml.org/">YAML</a> is a file format definition to represent structured data.
Unlike <a href="http://json.org/">JSON</a> which is derived from JavaScript,
YAML is independent (although, both are supported by a wide range languages).
It is also well readable by humans.</p>
<h1 id="python">Python</h1>
<p>Example file <a href="static/example.yaml">example.yaml</a> (see YAML homepage or Wikipedia for more)</p>
<div class="highlight"><pre><span class="l-Scalar-Plain">name</span><span class="p-Indicator">:</span> <span class="l-Scalar-Plain">YAML</span>
<span class="l-Scalar-Plain">note</span><span class="p-Indicator">:</span> <span class="s">"YAML</span><span class="nv"> </span><span class="s">Ain't</span><span class="nv"> </span><span class="s">Markup</span><span class="nv"> </span><span class="s">Language"</span>
<span class="l-Scalar-Plain">events</span><span class="p-Indicator">:</span>
<span class="p-Indicator">-</span> <span class="p-Indicator">{</span><span class="nv">date</span><span class="p-Indicator">:</span> <span class="nv">2004-01-29</span><span class="p-Indicator">,</span> <span class="nv">note</span><span class="p-Indicator">:</span> <span class="s">'Version</span><span class="nv"> </span><span class="s">1.0'</span><span class="p-Indicator">}</span>
<span class="p-Indicator">-</span> <span class="p-Indicator">{</span><span class="nv">date</span><span class="p-Indicator">:</span> <span class="nv">2005-01-18</span><span class="p-Indicator">,</span> <span class="nv">note</span><span class="p-Indicator">:</span> <span class="s">'Version</span><span class="nv"> </span><span class="s">1.1'</span><span class="p-Indicator">}</span>
<span class="p-Indicator">-</span> <span class="p-Indicator">{</span><span class="nv">date</span><span class="p-Indicator">:</span> <span class="nv">2009-10-01</span><span class="p-Indicator">,</span> <span class="nv">note</span><span class="p-Indicator">:</span> <span class="s">'Version</span><span class="nv"> </span><span class="s">1.2'</span><span class="p-Indicator">}</span>
</pre></div>
<p>Open this file and parse it into Python: </p>
<div class="highlight"><pre><span class="o">>>></span> <span class="kn">import</span> <span class="nn">yaml</span>
<span class="o">>>></span> <span class="n">f</span> <span class="o">=</span> <span class="nb">open</span><span class="p">(</span><span class="s">'examle.yaml'</span><span class="p">)</span>
<span class="o">>>></span> <span class="n">dataMap</span> <span class="o">=</span> <span class="n">safe_load</span><span class="p">(</span><span class="n">f</span><span class="p">)</span>
<span class="o">>>></span> <span class="n">f</span><span class="o">.</span><span class="n">close</span><span class="p">()</span>
<span class="o">>>></span> <span class="n">dataMap</span>
<span class="p">{</span><span class="s">'note'</span><span class="p">:</span> <span class="s">"YAML Ain't Markup Language"</span><span class="p">,</span> <span class="s">'name'</span><span class="p">:</span> <span class="s">'YAML'</span><span class="p">,</span> <span class="s">'events'</span><span class="p">:</span> <span class="p">[{</span><span class="s">'date'</span><span class="p">:</span>
<span class="n">datetime</span><span class="o">.</span><span class="n">date</span><span class="p">(</span><span class="mi">2004</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">29</span><span class="p">),</span> <span class="s">'note'</span><span class="p">:</span> <span class="s">'Version 1.0'</span><span class="p">},</span> <span class="p">{</span><span class="s">'date'</span><span class="p">:</span> <span class="n">datetime</span><span class="o">.</span><span class="n">date</span>
<span class="p">(</span><span class="mi">2005</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">18</span><span class="p">),</span> <span class="s">'note'</span><span class="p">:</span> <span class="s">'Version 1.1'</span><span class="p">},</span> <span class="p">{</span><span class="s">'date'</span><span class="p">:</span> <span class="n">datetime</span><span class="o">.</span><span class="n">date</span><span class="p">(</span><span class="mi">2009</span><span class="p">,</span> <span class="mi">10</span><span class="p">,</span> <span class="mi">1</span><span class="p">),</span>
<span class="s">'note'</span><span class="p">:</span> <span class="s">'Version 1.2'</span><span class="p">}]}</span>
<span class="o">>>></span> <span class="n">dataMap</span><span class="p">[</span><span class="s">'name'</span><span class="p">]</span>
<span class="s">'YAML'</span>
</pre></div>
<p>This is a Python dictionary.
To save a dictionary to a YAML file, use <code>yaml.dump(dataMap, f)</code> (<code>f</code> beeing a file stream opened for writing).
The <a href="http://stackoverflow.com/a/6866697/3274246">Source</a> gives an example how to convert this dictionary into a Python object:</p>
<div class="highlight"><pre><span class="o">>>></span> <span class="k">class</span> <span class="nc">MyStruct</span><span class="p">:</span>
<span class="o">...</span> <span class="k">def</span> <span class="nf">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="o">**</span><span class="n">entries</span><span class="p">):</span>
<span class="o">...</span> <span class="bp">self</span><span class="o">.</span><span class="n">__dict__</span><span class="o">.</span><span class="n">update</span><span class="p">(</span><span class="n">entries</span><span class="p">)</span>
<span class="o">...</span>
<span class="o">>>></span> <span class="n">y</span> <span class="o">=</span> <span class="n">MyStruct</span><span class="p">(</span><span class="o">**</span><span class="n">dataMap</span><span class="p">)</span>
<span class="o">>>></span> <span class="n">y</span><span class="o">.</span><span class="n">name</span>
<span class="s">'YAML'</span>
<span class="o">>>></span> <span class="n">y</span><span class="o">.</span><span class="n">events</span>
<span class="p">[{</span><span class="s">'date'</span><span class="p">:</span> <span class="n">datetime</span><span class="o">.</span><span class="n">date</span><span class="p">(</span><span class="mi">2004</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">29</span><span class="p">),</span> <span class="s">'note'</span><span class="p">:</span> <span class="s">'Version 1.0'</span><span class="p">},</span> <span class="p">{</span><span class="s">'date'</span><span class="p">:</span> <span class="n">datetime</span><span class="o">.</span><span class="n">date</span><span class="p">(</span><span class="mi">2005</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">18</span><span class="p">),</span>
<span class="s">'note'</span><span class="p">:</span> <span class="s">'Version 1.1'</span><span class="p">},</span> <span class="p">{</span><span class="s">'date'</span><span class="p">:</span> <span class="n">datetime</span><span class="o">.</span><span class="n">date</span><span class="p">(</span><span class="mi">2009</span><span class="p">,</span> <span class="mi">10</span><span class="p">,</span> <span class="mi">1</span><span class="p">),</span> <span class="s">'note'</span><span class="p">:</span> <span class="s">'Version 1.2'</span><span class="p">}]</span>
<span class="o">>>></span> <span class="n">y</span><span class="o">.</span><span class="n">events</span><span class="p">[</span><span class="mi">2</span><span class="p">]</span>
<span class="p">{</span><span class="s">'date'</span><span class="p">:</span> <span class="n">datetime</span><span class="o">.</span><span class="n">date</span><span class="p">(</span><span class="mi">2009</span><span class="p">,</span> <span class="mi">10</span><span class="p">,</span> <span class="mi">1</span><span class="p">),</span> <span class="s">'note'</span><span class="p">:</span> <span class="s">'Version 1.2'</span><span class="p">}</span>
<span class="o">>>></span> <span class="n">y</span><span class="o">.</span><span class="n">events</span><span class="p">[</span><span class="mi">2</span><span class="p">][</span><span class="s">'note'</span><span class="p">]</span>
<span class="s">'Version 1.2'</span>
</pre></div>
<h1 id="references">References</h1>
<ul>
<li><a href="http://stackoverflow.com/a/6866697/3274246">Stackoverflow</a></li>
</ul>Vim Copy and Paste2014-04-07T12:16:26+02:00Georg Wassentag:www.wassen.net,2014-04-07:vim-copy-and-paste.html<h1 id="command-mode-commands">Command mode commands</h1>
<p>The most important commands are <code>y</code> to yank (copy) and <code>p</code> to paste.
And not to forget <code>d</code> to delete.
The yank and delete commands put text into registers.
Without further information, the default register is used.</p>
<table>
<thead>
<tr>
<th>Command</th>
<th>Meaning</th>
</tr>
</thead>
<tbody>
<tr>
<td><code>yy</code></td>
<td>copy the current line to the default register</td>
</tr>
<tr>
<td><code>dd</code></td>
<td>delete the current line placing it in the default register</td>
</tr>
<tr>
<td><code>p</code></td>
<td>paste the content of the default register <em>right of</em> or <em>below</em> the cursor</td>
</tr>
<tr>
<td><code>P</code></td>
<td>paste the content of the default register <em>at the position</em> or <em>above</em> the cursor</td>
</tr>
<tr>
<td><code>"ayy</code></td>
<td>yank (the line) into named register <code>a</code></td>
</tr>
<tr>
<td><code>"ap</code></td>
<td>paste content of register <code>a</code></td>
</tr>
</tbody>
</table>
<p>see also: <code>:help copy-move</code></p>
<h1 id="special-registers">Special registers</h1>
<p>The following registers can be used like the named registers <code>a</code> to <code>Z</code>,
but they contain special information or have more functions.</p>
<table>
<thead>
<tr>
<th>Register</th>
<th>Function</th>
</tr>
</thead>
<tbody>
<tr>
<td><code>"*</code></td>
<td>X11 middle-mouse selection (copy by selecting with the right mouse button and paste with the middle mouse button)</td>
</tr>
<tr>
<td><code>"+</code></td>
<td>X11 clipboard (Edit -> Copy/Paste or Ctrl-X/C/V)</td>
</tr>
<tr>
<td><code>"/</code></td>
<td>last search term</td>
</tr>
<tr>
<td><code>":</code></td>
<td>last vi command (entered after <code>:</code> in the bottom line of the editor)</td>
</tr>
</tbody>
</table>Modern Bash Scripts2014-02-05T08:58:00+01:00Georg Wassentag:www.wassen.net,2014-02-05:modern-bash-scripts.html<p>So far, this is just a short selection of items.
I plan to convert this into a more elaborate form and provide examples sooner or later.</p>
<h1 id="general">General</h1>
<p>Bash has now many things built-in, that once required calling external commands.
Most notably:</p>
<ul>
<li>The testing of conditions used <code>[...]</code> which is just an alias for the program <code>test</code>.
Just using <code>[[...]]</code> instead lets Bash evaluate the condition itself.
This doesn't matter for single <code>if</code> statements, but complex scripts with loops
experience a huge speed-up.</li>
<li>Simple calculation and string handling can be done by <code>$(( 1 + I ))</code> (instead of <code>expr 1 + $I</code>)
and <code>${DIR/foo/bar}</code> instead of calling <code>sed</code>.</li>
<li>Regular Expressions are build in (use <code>[[...]]</code>)</li>
</ul>
<h1 id="small-bits-and-pieces">Small bits and pieces</h1>
<h2 id="camelcase">CamelCase</h2>
<p>For an auto-generated TeX file, I want to automatically generate a <code>CamelCase</code> expression
from filenames containing dashs and underlines.
Many suggestions on the internet propose using <code>sed</code> or other secondary tools,
but I tried to get it with Bash Buildins.</p>
<p>Bash has the <a href="http://www.gnu.org/software/bash/manual/html_node/Shell-Parameter-Expansion.html">parameter expansions</a>
<code>${parameter^pattern}</code> and <code>${parameter,pattern}</code> that convert the first matched character to upper or lower case.
By setting the pattern to <code>*</code> or <code>?</code>, the first letter of the variable <code>$parameter</code> is converted:</p>
<div class="highlight"><pre><span class="nv">$ STR</span><span class="o">=</span><span class="s1">'abc def'</span>
<span class="nv">$ </span><span class="nb">echo</span> <span class="si">${</span><span class="nv">STR</span><span class="p">^?</span><span class="si">}</span>
Abc def
<span class="nv">$ </span><span class="nb">echo</span> <span class="si">${</span><span class="nv">STR</span><span class="p">^*</span><span class="si">}</span>
Abc def
<span class="nv">$ </span><span class="nb">echo</span> <span class="si">${</span><span class="nv">STR</span><span class="p">^^*</span><span class="si">}</span>
ABC DEF
</pre></div>
<p>There are variants with two <code>^^</code> or <code>,,</code> that convert every occurence of a matched pattern.
But how to convert only the first letter of every word?
The solution is to use an array and apply the conversion to each element:</p>
<div class="highlight"><pre><span class="nv">$ ARR</span><span class="o">=(</span> <span class="nv">$STR</span> <span class="o">)</span>
<span class="nv">$ </span><span class="nb">echo</span> <span class="si">${</span><span class="nv">ARR</span><span class="p">[*]^*</span><span class="si">}</span>
Abc Def
</pre></div>
<p>Back to the initial challenge: convert a string like <code>my-filename_variant1</code>.
The approach is to convert the special characters to spaces
and then tokenize the character string at word boundaries to an <a href="http://www.gnu.org/software/bash/manual/html_node/Arrays.html">array</a>.</p>
<div class="highlight"><pre><span class="nv">$ TMP</span><span class="o">=</span><span class="s1">'my-filename_variant1'</span>
<span class="nv">$ TMP</span><span class="o">=</span><span class="si">${</span><span class="nv">TMP</span><span class="p">//-/ </span><span class="si">}</span> <span class="c"># convert all '-' to spaces</span>
<span class="nv">$ TMP</span><span class="o">=</span><span class="si">${</span><span class="nv">TMP</span><span class="p">//_/ </span><span class="si">}</span> <span class="c"># convert all '_' to spaces</span>
<span class="nv">$ TMP</span><span class="o">=(</span> <span class="nv">$TMP</span> <span class="o">)</span> <span class="c"># tokeninze words to array</span>
<span class="nv">$ TMP</span><span class="o">=</span><span class="si">${</span><span class="nv">TMP</span><span class="p">[*]^*</span><span class="si">}</span> <span class="c"># upper-case all words</span>
<span class="nv">$ TMP</span><span class="o">=</span><span class="si">${</span><span class="nv">TMP</span><span class="p">// /</span><span class="si">}</span> <span class="c"># remove spaces</span>
<span class="nv">$ </span><span class="nb">echo</span> <span class="nv">$TMP</span>
MyFilenameVariant1
</pre></div>
<p>There are probably more elegant ways with <code>sed</code>.
I'm not sure, if it could be solved with Bash's regular expressions.
And finally, I'm also not sure if this is still more efficient than a forked <code>sed</code>,
but it is amazing what can be done with Bash alone.</p>
<p>Source: solved this problem with the creative help of <a href="http://stackoverflow.com/a/11373345/3274246">Stackoverflow</a>.</p>Git Analyze2013-12-28T10:29:00+01:00Georg Wassentag:www.wassen.net,2013-12-28:git-analyze.html<p>Ever found an old copy of a repository and didn't know the purpose or state of if?
Was it just a test? Are there modifications that were not pushed anywhere?
When was it cloned in the first place? And when was is used the last time?</p>
<p>I didn't found any methods of Git itself or third-party tools to query this information
(okay, I didn't search very thoroughly).
This little script demonstrates what can be found when digging the .git directory.
The script is on <a href="https://github.com/georgwassen/Scripts/blob/master/gitanalyzer.sh">GitHub</a>.</p>
<h1 id="identify-a-github-repository">Identify a GitHub repository</h1>
<p>A GitHub repository carries its meta-information in a <code>.git</code> directory located
in its base directory. If the current working directory is a subdirectory,
the path must be followed towards the root directory.</p>
<div class="highlight"><pre><span class="nv">DIR</span><span class="o">=</span><span class="s1">''</span>
<span class="nv">ORIG_DIR</span><span class="o">=</span><span class="nv">$PWD</span>
<span class="k">if</span> <span class="o">[[</span> -d .git <span class="o">]]</span><span class="p">;</span> <span class="k">then</span>
<span class="nv">DIR</span><span class="o">=</span><span class="nv">$ORIG_DIR</span>
<span class="k">else</span>
<span class="k">while</span> <span class="o">[[</span> <span class="nv">$PWD</span> !<span class="o">=</span> / <span class="o">]]</span><span class="p">;</span> <span class="k">do</span>
<span class="nb">cd</span> ..
<span class="k">if</span> <span class="o">[[</span> -d .git <span class="o">]]</span><span class="p">;</span> <span class="k">then</span>
<span class="nv">DIR</span><span class="o">=</span><span class="nv">$PWD</span>
<span class="nb">break</span>
<span class="nb"> </span><span class="k">fi</span>
<span class="k">done</span>
<span class="k">fi</span>
<span class="k">if</span> <span class="o">[[</span>-z <span class="nv">$DIR</span> <span class="o">]]</span><span class="p">;</span> <span class="k">then</span>
<span class="nb">echo</span> <span class="s2">"ERROR: no .git directory found in path '</span><span class="nv">$ORIG_DIR</span><span class="s2">'"</span>
<span class="nb">exit</span>
<span class="k">fi</span>
</pre></div>
<p>These lines check if there is a <code>.git</code> directory in the current working directory
and if it can not be found, it steps upwards until either the root directory is reached
or a <code>.git</code> directory is found.
If the loop is left by the <code>while</code> condition, the variable <code>DIR</code> is still empty
and an error message is printed.</p>
<p>This code changes to the base directory of the Git repository.
Since it is a script with its own scope, it does not need to store the original
directory to restore it later. The calling environment is not changed.</p>
<h2 id="age-of-the-repo">Age of the repo</h2>
<p>The age of the repo can be derived from the oldest file in the .git directory.
This is probably not the most stable algorithm. Later, the <code>logs</code> directory is
explained that holds a better source for this information.</p>
<div class="highlight"><pre><span class="c"># try to derive the age (date of init or clone) from .git files</span>
<span class="c"># (use oldest file in .git directory)</span>
<span class="nb">echo</span> -n <span class="s2">"init'ed or clone'd most probably on: "</span>
stat -c <span class="s1">'%Y %y %n'</span> <span class="nv">$DIR</span>/.git/* <span class="p">|</span> sort <span class="p">|</span> head -n1 <span class="p">|</span> awk <span class="s1">'{print $2 " " $3 " (" $5 ")" }'</span>
</pre></div>
<p>The <code>stat</code> utility is given a format string to print the modification date
as seconds since the UNIX epoch, in a readable format and the file name.
This list is sorted, the first line is extracted and the columns 2 and 3 (date, time) and 5 (filename)
are printed.</p>
<h1 id="simple-information-by-git-tools">Simple information by <code>git</code> tools</h1>
<p>A basic information is the <code>.git/description</code> that can be set for every repository.
It appears not to be used by the <code>git</code> tools, but might be read by other tools (GitWeb)
or hooks.
If the file exists and does not contain the default ("Unnamed repository..."),
its content is printed.</p>
<h2 id="remote-links">Remote links</h2>
<p>The remote links are printed by <code>git remote -v</code>, but the lines are annotated with (fetch) and (push).
To just see the links, the second column is cut out and sorted and unified:</p>
<div class="highlight"><pre><span class="nb">echo</span> -n <span class="s2">"Remote links: "</span>
git remote -v <span class="p">|</span> cut -f2 <span class="p">|</span> cut -d<span class="s1">' '</span> -f1 <span class="p">|</span> sort <span class="p">|</span> uniq
</pre></div>
<h2 id="svn-connection">SVN connection</h2>
<p>The command <code>git svn info</code> should reveal any ties to a Subversion repository.
If the command gives an error, the output is suppressed:</p>
<div class="highlight"><pre><span class="nv">SVNINFO</span><span class="o">=</span><span class="k">$(</span>git svn info 2><span class="p">&</span>1<span class="k">)</span>
<span class="k">if</span> <span class="o">[[</span> ! <span class="s2">"</span><span class="nv">$SVNINFO</span><span class="s2">"</span> <span class="o">=</span>~ ^Unable <span class="o">]]</span><span class="p">;</span> <span class="k">then</span>
<span class="nb">echo</span> -n <span class="s2">"Git-SVN info: "</span> <span class="nv">$SVNINFO</span>
<span class="k">fi</span>
</pre></div>
<h2 id="last-commit">Last commit</h2>
<p>The commit log can be flexibly formated with <code>git log</code>. The script uses:</p>
<div class="highlight"><pre><span class="nb">echo</span> -n <span class="s2">"Last commit: "</span>
git --no-pager log --all -n1 --format<span class="o">=</span><span class="s2">"</span><span class="si">${</span><span class="nv">COLGITHASH</span><span class="si">}</span><span class="s2">%h </span><span class="si">${</span><span class="nv">COLGITDATE</span><span class="si">}</span><span class="s2">%ci</span><span class="si">${</span><span class="nv">COLGITRESET</span><span class="si">}</span><span class="s2">%d </span><span class="si">${</span><span class="nv">COLGITSUBJECT</span><span class="si">}</span><span class="s2">%s</span><span class="si">${</span><span class="nv">COLGITRESET</span><span class="si">}</span><span class="s2">"</span>
</pre></div>
<p>The variables <code>$COLGIT*</code> contain the <code>git log</code> format <code>%C(...)</code> as documented in its man-pages.
The can be set empty to suppress color (see the final script linked above for the whole picture).</p>
<h1 id="git-logs">Git logs</h1>
<p>The directory <code>.git/logs</code> holds files with a history log for various objects.
The <code>HEAD</code> contains the initialization or cloning, any pushes, fetches and pulls
and commits and checkouts.</p>
<p>The first line gives the information when the repo was creates by initialization
or cloning. In the latter case, the clone source is given.
Depending on verbosity, the script prints either the first and last entry or the full
history.
The tokenization of the lines is a bit tricky because after two hashes, the
name can be one or multiple words. The E-Mail address is enclosed in angle brackets.
Then follows a UNIX time stamp (seconds since epoch), the time zone and a description
of the action.</p>
<div class="highlight"><pre><span class="k">function</span> tokenize_log<span class="o">()</span>
<span class="o">{</span>
<span class="nb">read </span>REV0 REV1 REST < <<span class="o">(</span><span class="nb">echo</span> <span class="s2">"</span><span class="nv">$@</span><span class="s2">"</span><span class="o">)</span> <span class="c"># prev. and current revision sha1, remainder</span>
echo_v3 <span class="s2">"REV0 = '</span><span class="nv">$REV0</span><span class="s2">'"</span> <span class="c"># echo_v3 prints only on verbosity >= 3</span>
echo_v3 <span class="s2">"REV1 = '</span><span class="nv">$REV1</span><span class="s2">'"</span>
<span class="nv">NAME</span><span class="o">=</span><span class="si">${</span><span class="nv">REST</span><span class="p">%% <*</span><span class="si">}</span> <span class="c"># Name is up to first angle bracket</span>
echo_v3 <span class="s2">"NAME = '</span><span class="nv">$NAME</span><span class="s2">'"</span>
<span class="nv">REST</span><span class="o">=</span><span class="si">${</span><span class="nv">REST</span><span class="p">##* <</span><span class="si">}</span> <span class="c"># Remainder is after first bracket</span>
<span class="nv">MAIL</span><span class="o">=</span><span class="si">${</span><span class="nv">REST</span><span class="p">%%>*</span><span class="si">}</span> <span class="c"># Mail is up to closing angle bracket</span>
echo_v3 <span class="s2">"MAIL = '</span><span class="nv">$MAIL</span><span class="s2">'"</span>
<span class="nv">REST</span><span class="o">=</span><span class="si">${</span><span class="nv">REST</span><span class="p">##*> </span><span class="si">}</span> <span class="c"># Remainder is after angle bracket</span>
<span class="nb">read </span>TIME ZONE ACTION < <<span class="o">(</span><span class="nb">echo</span> <span class="nv">$REST</span> <span class="o">)</span> <span class="c"># Time, Zone, Action (multiple words)</span>
echo_v3 <span class="s2">"TIME = '</span><span class="nv">$TIME</span><span class="s2">'"</span>
echo_v3 <span class="s2">"ZONE = '</span><span class="nv">$ZONE</span><span class="s2">'"</span>
<span class="nv">DATE</span><span class="o">=</span><span class="k">$(</span>date -d@<span class="nv">$TIME</span> +<span class="s1">'%Y-%m-%d %H:%M:%S'</span><span class="k">)</span> <span class="c"># convert UNIX time stamp into readable date</span>
echo_v3 <span class="s2">"ACTION = '</span><span class="nv">$ACTION</span><span class="s2">'"</span>
<span class="k">if</span> <span class="o">[[</span> -z <span class="nv">$ACTION</span> <span class="o">]]</span><span class="p">;</span> <span class="k">then</span>
<span class="nv">ACTION</span><span class="o">=</span><span class="s2">"git init"</span> <span class="c"># if no action: it was a `git init`</span>
<span class="k">fi</span>
<span class="o">}</span>
<span class="k">if</span> <span class="o">[[</span> <span class="nv">$PARAM_VERBOSE</span> -ge <span class="m">1</span> <span class="o">]]</span><span class="p">;</span> <span class="k">then</span>
<span class="c"># full history: loop over all lines of .git/refs/heads/master</span>
<span class="nb">echo</span> -e <span class="s2">"History of master"</span>
<span class="k">while</span> <span class="nb">read </span>LINE<span class="p">;</span> <span class="k">do</span>
<span class="c"># function fills global variables $REV0, $REV1, ..., $ACTION, $DATE, $NAME, $MAIL</span>
tokenize_log <span class="s2">"</span><span class="nv">$LINE</span><span class="s2">"</span>
<span class="c"># print in a convenient format</span>
<span class="nb">echo</span> -e <span class="s2">" </span><span class="nv">$ACTION</span><span class="s2"> on </span><span class="nv">$DATE</span><span class="s2"> by </span><span class="nv">$NAME</span><span class="s2"> </span><span class="nv">$MAIL</span><span class="s2">"</span>
<span class="k">done</span> < <<span class="o">(</span>cat <span class="nv">$DIR</span>/.git/logs/refs/heads/master<span class="o">)</span>
<span class="k">else</span>
<span class="c"># print only the first and last line of .git/logs/HEAD</span>
tokenize_log <span class="k">$(</span>head -n1 <span class="nv">$DIR</span>/.git/logs/HEAD<span class="k">)</span>
<span class="nb">echo</span> -e <span class="s2">"Source of this Repo: </span><span class="nv">$ACTION</span><span class="s2"> on </span><span class="nv">$DATE</span><span class="s2">"</span>
tokenize_log <span class="k">$(</span>tail -n1 <span class="nv">$DIR</span>/.git/logs/HEAD<span class="k">)</span>
<span class="nb">echo</span> -e <span class="s2">"Last action: </span><span class="nv">$ACTION</span><span class="s2"> on </span><span class="nv">$DATE</span><span class="s2"> by </span><span class="nv">$NAME</span><span class="s2"> </span><span class="nv">$MAIL</span><span class="s2">"</span>
<span class="k">fi</span>
</pre></div>
<p>The function <code>tokenize_log</code> receives a line of the logfile and fills the
global variables. The <code>echo_v3</code> function only prints the debug output,
if the log level (verbosity) is above or equal to three.</p>
<p>The same output is created for all remotes:</p>
<div class="highlight"><pre><span class="k">for</span> D in <span class="nv">$DIR</span>/.git/logs/refs/remotes/*<span class="p">;</span> <span class="k">do</span>
<span class="nv">REMOTE</span><span class="o">=</span><span class="si">${</span><span class="nv">D</span><span class="p">##*/</span><span class="si">}</span> <span class="c"># extract last part of path</span>
<span class="c"># get name of server</span>
<span class="nv">REMSERVER</span><span class="o">=</span><span class="k">$(</span>git remote -v <span class="p">|</span> grep <span class="nv">$REMOTE</span> <span class="p">|</span> cut -f2 <span class="p">|</span> cut -d<span class="s1">' '</span> -f1 <span class="p">|</span> sort <span class="p">|</span> uniq<span class="k">)</span>
<span class="nb">echo</span> -e <span class="s2">"History of </span><span class="nv">$REMOTE</span><span class="s2"> (</span><span class="nv">$REMSERVER</span><span class="s2">)"</span>
<span class="k">if</span> <span class="o">[[</span> <span class="nv">$PARAM_VERBOSE</span> -ge <span class="m">1</span> <span class="o">]]</span><span class="p">;</span> <span class="k">then</span>
<span class="c"># full history</span>
<span class="k">while</span> <span class="nb">read </span>LINE<span class="p">;</span> <span class="k">do</span>
tokenize_log <span class="s2">"</span><span class="nv">$LINE</span><span class="s2">"</span>
<span class="nb">echo</span> -e <span class="s2">" </span><span class="nv">$ACTION</span><span class="s2"> on </span><span class="nv">$DATE</span><span class="s2"> by </span><span class="nv">$NAME</span><span class="s2"> </span><span class="nv">$MAIL</span><span class="s2">"</span>
<span class="k">done</span> < <<span class="o">(</span>cat <span class="nv">$D</span>/master<span class="o">)</span>
<span class="k">else</span>
<span class="c"># only first and last entry</span>
tokenize_log <span class="k">$(</span>head -n1 <span class="nv">$D</span>/master<span class="k">)</span>
<span class="nb">echo</span> -e <span class="s2">" First action: </span><span class="nv">$ACTION</span><span class="s2"> on </span><span class="nv">$DATE</span><span class="s2">"</span>
tokenize_log <span class="k">$(</span>tail -n1 <span class="nv">$D</span>/master<span class="k">)</span>
<span class="nb">echo</span> -e <span class="s2">" Last action: </span><span class="nv">$ACTION</span><span class="s2"> on </span><span class="nv">$DATE</span><span class="s2"> by </span><span class="nv">$NAME</span><span class="s2"> </span><span class="nv">$MAIL</span><span class="s2">"</span>
<span class="k">fi</span>
<span class="k">done</span>
</pre></div>Inline Assembler2013-12-17T13:29:00+01:00Georg Wassentag:www.wassen.net,2013-12-17:inline-assembler.html<p>Assembly instructions can be embedded in C code with the <code>asm</code> keyword:</p>
<div class="highlight"><pre><span class="n">asm</span> <span class="p">(</span><span class="s">"mov ..."</span><span class="p">);</span>
</pre></div>
<p>Multiple lines must be separated by newlines:</p>
<div class="highlight"><pre><span class="n">asm</span> <span class="p">(</span><span class="s">"mov ...</span><span class="se">\n</span><span class="s">"</span>
<span class="s">"add ...</span><span class="se">\n</span><span class="s">"</span>
<span class="s">"..."</span><span class="p">);</span>
</pre></div>
<p>The assembly instructions must be provided either in <em>Intel</em> or <em>AT&T</em> syntax
which will be explained below.
Some compilers or C standards require the keyword <code>__asm</code> or <code>__asm__</code> (two underline characters).
The keyword <code>volatile</code> will instruct the compiler not to change or remove an inline assembly block
for example to prohibit the compiler from removing an empty loop:</p>
<div class="highlight"><pre><span class="k">for</span> <span class="p">(</span><span class="kt">int</span> <span class="n">i</span> <span class="o">=</span> <span class="mi">0</span><span class="p">;</span> <span class="n">i</span> <span class="o"><</span> <span class="mi">100000</span><span class="p">;</span> <span class="n">i</span><span class="o">++</span><span class="p">)</span> <span class="p">{</span>
<span class="n">asm</span> <span class="k">volatile</span> <span class="p">(</span><span class="s">"nop"</span><span class="p">);</span>
<span class="p">}</span>
</pre></div>
<p>Under some circumstances, it is possible to use C variables in the assembly code
(since the labels are known to both the compiler and assembler).</p>
<h1 id="gcc">GCC</h1>
<p>The GNU compiler collection does not understand the embedded assembly and just
prints it to the assembly code it generates and that is further translated to machine instructions
by the GNU assembler.
Since GCC uses the <em>AT&T</em> syntax (probably since it is easier to machine-generate),
the embedded code must be in that syntax.
However, the compiler (and subsequently, the assembler) can be configured
to use and understand the <em>Intel</em> syntax:</p>
<div class="highlight"><pre><span class="nv">$ </span>gcc -masm<span class="o">=</span>intel
</pre></div>
<p>This instructs the compiler to generate <em>Intel</em> syntax assembly and consequently
also calls the assembler with that parameter.
Now, the inline assembly can be in the (to me) more natural <em>Intel</em> syntax.</p>
<h2 id="extended-inline-assembly">Extended Inline Assembly</h2>
<p>To help the GCC to correctly embed the piece of assembly into its own generated code,
it must be provided with some information about the assembly:</p>
<ul>
<li><em>input</em>: values to load into registers before the execution of this block</li>
<li><em>output</em>: where should the register values go after the execution</li>
<li><em>clobbered</em>: other registers, that change their content so that they must be preserved
(push on the stack before and restore after execution)</li>
</ul>
<p>The complete syntax is:</p>
<div class="highlight"><pre><span class="n">asm</span> <span class="p">(</span><span class="s">"mov ..."</span> <span class="o">:</span> <span class="cm">/*output*/</span> <span class="o">:</span> <span class="cm">/*input*/</span> <span class="o">:</span> <span class="cm">/*clobbered*/</span> <span class="p">);</span>
</pre></div>
<p>The <em>output</em> and <em>input</em> registers are provided with the information
what goes where.
In the following example, <code>=c</code> means: "the new value of ECX", <code>a</code> means: "load EAX with ...".
The full list of those modifiers is in the documentation linked below.</p>
<div class="highlight"><pre><span class="kt">uint32_t</span> <span class="n">func</span> <span class="o">=</span> <span class="mi">1</span><span class="p">;</span>
<span class="kt">uint32_t</span> <span class="n">reg_ecx</span><span class="p">,</span> <span class="n">reg_edx</span><span class="p">;</span>
<span class="n">asm</span> <span class="nf">volatile</span> <span class="p">(</span> <span class="s">"cpuid"</span>
<span class="o">:</span> <span class="s">"=c"</span><span class="p">(</span><span class="n">reg_ecx</span><span class="p">),</span> <span class="s">"=d"</span><span class="p">(</span><span class="n">reg_edx</span><span class="p">)</span> <span class="cm">/* output: ECX and EDX */</span>
<span class="o">:</span> <span class="s">"a"</span><span class="p">(</span><span class="n">func</span><span class="p">)</span> <span class="cm">/* input: func into EAX */</span>
<span class="o">:</span> <span class="s">"ebx"</span> <span class="p">);</span> <span class="cm">/* also modified: EBX */</span>
</pre></div>
<h2 id="references">References</h2>
<ul>
<li><a href="http://gcc.gnu.org/onlinedocs/gcc/Extended-Asm.html">GNU doc: Assembler Instructions with C Expression Operands</a></li>
<li><a href="http://gcc.gnu.org/onlinedocs/gcc/Constraints.html">GNU doc: Constraints</a></li>
</ul>
<h1 id="intel-syntax">Intel syntax</h1>
<p>The <em>Intel</em> Syntax the natural one (used in most textbooks and the processor manuals from Intel and AMD).
The major property is the <em>target</em>, <em>source</em> sequence (like in the assignment of a variable in C):</p>
<div class="highlight"><pre><span class="nf">MOV</span> <span class="nb">eax</span><span class="p">,</span> <span class="nb">ebx</span> <span class="c1">; EAX := EBX</span>
<span class="nf">ADD</span> <span class="nb">ebx</span><span class="p">,</span> <span class="mi">10</span> <span class="c1">; EBX += 10</span>
</pre></div>
<p>Example program:</p>
<div class="highlight"><pre><span class="cp">#include <stdio.h></span>
<span class="kt">int</span> <span class="n">x</span> <span class="o">=</span> <span class="mi">5</span><span class="p">;</span>
<span class="kt">int</span> <span class="nf">main</span><span class="p">(</span><span class="kt">int</span> <span class="n">argc</span><span class="p">,</span> <span class="kt">char</span> <span class="o">**</span><span class="n">argv</span><span class="p">)</span> <span class="p">{</span>
<span class="n">printf</span><span class="p">(</span><span class="s">"x = %u</span><span class="se">\n</span><span class="s">"</span><span class="p">,</span> <span class="n">x</span><span class="p">);</span>
<span class="n">asm</span> <span class="p">(</span>
<span class="s">"mov eax, x"</span> <span class="s">"</span><span class="se">\r\n</span><span class="s">"</span>
<span class="s">"mov ecx, eax"</span> <span class="s">"</span><span class="se">\r\n</span><span class="s">"</span>
<span class="s">"add ecx, eax"</span> <span class="s">"</span><span class="se">\r\n</span><span class="s">"</span>
<span class="s">"mov x, ecx"</span> <span class="s">"</span><span class="se">\r\n</span><span class="s">"</span>
<span class="o">:::</span> <span class="s">"eax"</span><span class="p">,</span> <span class="s">"ecx"</span>
<span class="p">);</span>
<span class="n">printf</span><span class="p">(</span><span class="s">"x = %u</span><span class="se">\n</span><span class="s">"</span><span class="p">,</span> <span class="n">x</span><span class="p">);</span>
<span class="k">return</span> <span class="mi">0</span><span class="p">;</span>
<span class="p">}</span>
</pre></div>
<p>This program compiles with <code>gcc -masm=intel</code>.
If using <code>gcc -S -o - -masm=intel filename.c</code>, the assembly generated by the compiler is printed
in <em>Intel</em> syntax to the screen.</p>
<p>Theoretically, both syntaxes can be mixed by using the assembly keyword <code>.intel_syntax noprefix</code>
and <code>.att_syntax noprefix</code>. But when doing so, don't forget to restore the <em>AT&T</em> setting at the
end of <strong>every</strong> inline assembly block.</p>
<h1 id="att-syntax">AT&T syntax</h1>
<p>The GCC uses this syntax by default.
It has the sequence <em>source</em>, <em>target</em>.
Further, the instructions are postfixed with a modifier indicating the bit-width (here: <code>l</code> for long=32 bit)
and constant numbers must be prepended with a <code>$</code> sign.</p>
<div class="highlight"><pre><span class="nf">movl</span> <span class="nv">%ebx</span><span class="p">,</span> <span class="nv">%eax</span> <span class="c"># EAX -> EBX</span>
<span class="nf">addl</span> <span class="no">$10</span><span class="p">,</span> <span class="nv">%ebx</span> <span class="c"># (EBX + 10) -> EBX</span>
</pre></div>Git-Subversion2013-12-17T10:56:00+01:00Georg Wassentag:www.wassen.net,2013-12-17:git-subversion.html<p>This article shows how to import a Subversion repository to <a href="http://www.wassen.net/git.html">Git</a>.</p>
<h1 id="import-subversion-repository">Import Subversion Repository</h1>
<p>based on a <a href="http://john.albin.net/git/convert-subversion-to-git">migration tutorial</a></p>
<ol>
<li>
<p>Prepare a file of users (Edit the lines to match <code>uname = Firstname Lastname <email@example.com></code>)</p>
<div class="highlight"><pre><span class="nv">$ </span>svn log -q <span class="p">|</span> awk -F <span class="s1">'|'</span> <span class="s1">'/^r/ {sub("^ ", "", $2); sub(" $", "", $2); print $2" = "$2" <"$2">"}'</span> <span class="p">|</span> sort -u > authors-transform.txt
<span class="nv">$ </span>vim authors-transform.txt
</pre></div>
</li>
<li>
<p>Checkout the Subversion repository</p>
<div class="highlight"><pre><span class="nv">$ </span>git svn clone https://svn.example.com/svn/baserepo/some/subdirectory --no-metadata -A ../authors-transform.txt
</pre></div>
<ul>
<li><code>--no-metadata</code> : if that's a one-time import, this waives the linkage of Git Commits to Subversion Revisions.
If you want to update the Git Repo after further Subversion Commits, don't use this parameter!</li>
<li><code>-A ../authors-transform.txt</code> : used to translate Subversion usernames to Git's Name/E-Mail commit entries.</li>
</ul>
<p>This uses the previously created authors file
to convert Subversion accounts to names and E-Mail addresses used in Git commits.</p>
</li>
<li>
<p>If you have set <code>svn:ignore</code> properties, convert them to <code>.gitignore</code></p>
<div class="highlight"><pre><span class="nv">$ </span>git svn show-ignore > .gitignore
<span class="nv">$ </span>git add .gitignore
<span class="nv">$ </span>git commit -m <span class="s2">"Convert svn:ignore properties to .gitignore"</span>
</pre></div>
</li>
<li>
<p>Bring the Repository to a Server.
Here are two alternatives.</p>
<ol>
<li>
<p>Create an attatched Git repository that can pull in (and merge?) later changes from Subversion (untested so far).
Don't use the <code>--no-metadata</code> parameter to <code>git svn clone</code> (Step 2) in this case!</p>
<ul>
<li>
<p>Create a repository on a server (or locally) to push to</p>
<div class="highlight"><pre><span class="nv">$ </span>ssh git@www.example.com
<span class="nv">$ </span>mkdir -p git/MyNewRepo.git
<span class="nv">$ </span><span class="nb">cd </span>git/MyNewRepo.git
<span class="nv">$ </span>git init --bare
<span class="nv">$ </span><span class="nb">exit</span>
</pre></div>
</li>
<li>
<p>Then add the new repository to the Subversion checkout</p>
<div class="highlight"><pre><span class="nv">$ </span>git remote add bare git@www.example.com:git/MyNewRepo.git
<span class="nv">$ </span>git config remote.bare.push <span class="s1">'refs/remotes/*:refs/heads/*'</span>
<span class="nv">$ </span>git push bare
<span class="nv">$ </span>git push bare master
</pre></div>
</li>
</ul>
<p>The <em>refspec</em> in the second command configures a <code>git push bare</code> (without explicit branch)
to always push all remote branches to this remote repo.</p>
</li>
<li>
<p>This proceeding is to create a detatched once-only import to move a repository from Subversion over to Git.
This uses a temporary Git repo (called <code>git-svn</code>) that pushes to a server.
By doing so, the connection to Subversion gets lost
(this can most probably also be achieved by other ways...).</p>
<ul>
<li>
<p>The new repo was created on Github and cloned to the local hard disk.
This create a clean connection between the local clone and the upstream repo.</p>
</li>
<li>
<p>Then I had two Git repos (and the old Subversion):
The <code>new.git</code> is a clone of the Github repo and is correctly tied to that one (for pushing).</p>
<div class="highlight"><pre>checkout/<span class="nv">$ </span>ls
new.git old.git-svn old.svn
</pre></div>
</li>
<li>
<p>In the Git-Svn repo, add the <code>new.git</code> as remote repo with the name <code>myclone</code>:</p>
<div class="highlight"><pre>checkout/old.git-svn/<span class="nv">$ </span>git remote add myclone ../new.git
checkout/old.git-svn/<span class="nv">$ </span>git config remote.myclone.push <span class="s1">'refs/remotes/*:refs/heads/*'</span>
checkout/old.git-svn/<span class="nv">$ </span>git push myclone
</pre></div>
</li>
<li>
<p>This results in <code>new.git</code> having the new branch <code>git-svn</code> that can be merged to the <code>master</code> branch:</p>
<div class="highlight"><pre>checkout/new.git/<span class="nv">$ </span>git branch
git-svn
*master
checkout/new.git/<span class="nv">$ </span>git checkout git-svn
checkout/new.git/<span class="nv">$ </span>ls
checkout/new.git/<span class="nv">$ </span>git checkout master
checkout/new.git/<span class="nv">$ </span>git merge git-svn
checkout/new.git/<span class="nv">$ </span>git branch -d git-svn
</pre></div>
</li>
<li>
<p>Check everything, and if it's fine, push the new <code>master</code> branch to the Server.</p>
</li>
</ul>
</li>
</ol>
</li>
<li>
<p>There are more steps to follow in the original tutorial,
especially if using the default Subversion layout with <code>trunk</code>, <code>branches</code> and <code>tags</code>.
Check if everything works, then you can delete the git-svn checkout.</p>
</li>
</ol>Parsing Command-Line Parameters2013-12-10T19:45:00+01:00Georg Wassentag:www.wassen.net,2013-12-10:parsing-command-line-parameters.html<h1 id="basic">Basic</h1>
<p>The parameters are in the variables <code>$1</code>, <code>$2</code>, etc. The special variable <code>$#</code> contains
the number of parameters. Further, <code>$0</code> contains the name of the script how it was called
by the user.</p>
<p>If there are more than 9 parameters or to process the parameters one by one,
the build-in command <code>shift</code> moves all parameters to the next lower variable.</p>
<p><a href="static/param1.sh">param1.sh</a></p>
<div class="highlight"><pre><span class="c">#!/bin/bash</span>
<span class="nb">echo</span> <span class="nv">$1</span> <span class="nv">$2</span> <span class="nv">$3</span>
<span class="nb">shift</span>
<span class="nb">echo</span> <span class="nv">$1</span> <span class="nv">$2</span> <span class="nv">$3</span>
</pre></div>
<p>executed with parameters "a", "b", and "c":</p>
<div class="highlight"><pre><span class="nv">$ </span>./param1.sh a b c
a b c
b c
</pre></div>
<p>This can be used for simple processing of multiple input options:</p>
<p><a href="static/param2.sh">param2.sh</a></p>
<div class="highlight"><pre><span class="nb">echo</span> <span class="nv">$@</span>
<span class="k">while</span> <span class="o">[[</span> <span class="nv">$# </span>-gt <span class="m">0</span> <span class="o">]]</span><span class="p">;</span> <span class="k">do</span>
<span class="nb">echo</span> <span class="nv">$1</span>
<span class="nb">shift</span>
<span class="k">done</span>
</pre></div>
<p>The first line prints all parameters (another special variable),
then the <code>while</code> loop prints <code>$1</code> and shifts the remaining parameters
until the number of parameters <code>$#</code> is 0.</p>
<div class="highlight"><pre><span class="nv">$ </span>./param2.sh a b c
a b c
a
b
c
</pre></div>
<p>This can be extended with <code>if</code> or <code>case</code> to check for actual parameters
(examples for that are in the following examples).</p>
<p>The major advantage of the tools presented in the following are their
normalizing capabilities. If multiple switches (parameters without argument)
are given together as <code>-abc</code>, the normalizing rewrites them to <code>-a -b -c</code>.</p>
<h1 id="getopt">Getopt</h1>
<p><code>getopt</code> is a binary that parses the parameters and creates a unified (normalized)
string that can be parsed.
The line <code>eval set -- "$ARGS"</code> sets the parameters of the script to the normalized
version processed by <code>getopt</code>.</p>
<p><a href="static/param3.sh">param3.sh</a></p>
<div class="highlight"><pre><span class="nv">ARGS</span><span class="o">=</span><span class="k">$(</span>getopt -o <span class="s1">'n:v::h'</span> -l <span class="s1">'help'</span> -- <span class="s2">"</span><span class="nv">$@</span><span class="s2">"</span><span class="k">)</span> <span class="c"># parse parameters and store normalized string in $ARGS</span>
<span class="nb">eval set</span> -- <span class="s2">"</span><span class="nv">$ARGS</span><span class="s2">"</span><span class="p">;</span> <span class="c"># set parameters to preprocessed string $ARGS</span>
<span class="k">while</span> <span class="o">[[</span> <span class="nv">$# </span>-gt <span class="m">0</span> <span class="o">]]</span><span class="p">;</span> <span class="k">do</span>
<span class="nb">echo</span> -n <span class="s2">"[</span><span class="nv">$1</span><span class="s2">] "</span>
<span class="k">case</span> <span class="s2">"</span><span class="nv">$1</span><span class="s2">"</span> in
-n<span class="o">)</span>
<span class="nb">echo</span> <span class="s2">"found parameter -n"</span>
<span class="nb">echo</span> <span class="s2">" required parameter is '</span><span class="nv">$2</span><span class="s2">'"</span>
<span class="nb">shift</span> <span class="c"># remove required parameter</span>
<span class="p">;;</span>
-v<span class="o">)</span>
<span class="nb">echo</span> <span class="s2">"that's a -v"</span>
<span class="k">if</span> <span class="o">[[</span> -n <span class="nv">$2</span> <span class="o">]]</span><span class="p">;</span> <span class="k">then</span>
<span class="nb">echo</span> <span class="s2">" optional parameter '</span><span class="nv">$2</span><span class="s2">'"</span>
<span class="k">else</span>
<span class="nb">echo</span> <span class="s2">" no optional parameter provided"</span>
<span class="k">fi</span>
<span class="nb">shift</span> <span class="c"># remove optional parameter </span>
<span class="c"># (was inserted as /empty/ if not provided by user</span>
<span class="p">;;</span>
-h<span class="p">|</span>--help<span class="o">)</span>
<span class="nb">echo</span> <span class="s2">"someone asked for help with -h or --help"</span>
<span class="p">;;</span>
--<span class="o">)</span>
<span class="nb">echo</span> <span class="s2">"that was the last option, following are free parameters"</span>
<span class="p">;;</span>
*<span class="o">)</span>
<span class="nb">echo</span> <span class="s2">"thats an unknown parameter"</span>
<span class="k">esac</span>
<span class="nb">shift</span>
<span class="k">done</span>
</pre></div>
<p>The optstring (after <code>-o</code>) defines the valid parameters. Like in the C library function <code>getopt()</code>,
the parameters can have an optional argument (two colons) or a required one (one colon).
In the example above, the <code>-n</code> must be given an argument, the <code>-v</code> has an optional argument.
The <code>-h</code> and <code>--help</code> never expect an argument.</p>
<div class="highlight"><pre><span class="nv">$ </span>./param3.sh -n3 -v -v3 -h --help
<span class="o">[</span>-n<span class="o">]</span> found parameter -n
required parameter is <span class="s1">'3'</span>
<span class="o">[</span>-v<span class="o">]</span> that<span class="s1">'s a -v</span>
<span class="s1"> no optional parameter provided</span>
<span class="s1">[-v] that'</span>s a -v
optional parameter <span class="s1">'3'</span>
<span class="o">[</span>-h<span class="o">]</span> someone asked <span class="k">for</span> <span class="nb">help </span>with -h or --help
<span class="o">[</span>--help<span class="o">]</span> someone asked <span class="k">for</span> <span class="nb">help </span>with -h or --help
<span class="o">[</span>--<span class="o">]</span> that was the last option, following are free parameters
</pre></div>
<p>See the man-page to <code>getopt</code> for more details.</p>
<h1 id="getopts">Getopts</h1>
<p>This is a build-in of Bash. And as such, it is somehow easier and more straightforward
to use as it can directly be used in the <code>while</code> loop:</p>
<p><a href="static/param4.sh">param4.sh</a></p>
<div class="highlight"><pre><span class="k">while</span> <span class="nb">getopts </span>n:v::h PARAM<span class="p">;</span> <span class="k">do</span>
<span class="nb">echo</span> -n <span class="s2">"[</span><span class="nv">$PARAM</span><span class="s2">] "</span>
<span class="k">case</span> <span class="s2">"</span><span class="nv">$PARAM</span><span class="s2">"</span> in
n<span class="o">)</span>
<span class="nb">echo</span> <span class="s2">"found parameter -n"</span>
<span class="nb">echo</span> <span class="s2">" required parameter is '</span><span class="nv">$OPTARG</span><span class="s2">'"</span>
<span class="p">;;</span>
v<span class="o">)</span>
<span class="nb">echo</span> <span class="s2">"that's a -v"</span>
<span class="k">if</span> <span class="o">[[</span> -n <span class="nv">$OPTARG</span> <span class="o">]]</span><span class="p">;</span> <span class="k">then</span>
<span class="nb">echo</span> <span class="s2">" optional parameter '</span><span class="nv">$OPTARG</span><span class="s2">'"</span>
<span class="k">else</span>
<span class="nb">echo</span> <span class="s2">" no optional parameter provided"</span>
<span class="k">fi</span>
<span class="p">;;</span>
h<span class="o">)</span>
<span class="nb">echo</span> <span class="s2">"someone asked for help with -h"</span>
<span class="p">;;</span>
<span class="k">esac</span>
<span class="k">done</span>
<span class="c"># additional free arguments:</span>
<span class="nb">shift</span> <span class="k">$((</span> OPTIND <span class="o">-</span> <span class="m">1</span><span class="k">))</span>
<span class="nb">echo</span> <span class="s2">"more parameters: '</span><span class="nv">$@</span><span class="s2">'"</span>
</pre></div>
<p>The changes are:
* <code>getopts</code> is directly put in the <code>while</code> loop and puts its result in <code>$PARAM</code>.
* It does not support long options (e.g. <code>--help</code>)
* The cases are only the letter while <code>getopt</code> included the dash.
* Additional non-option (free) arguments remain unread and can be processed
later (after shifting all processed parameters).</p>Bibgrep and Texgrep2013-12-10T13:08:00+01:00Georg Wassentag:www.wassen.net,2013-12-10:bibgrep-and-texgrep.html<p>If you enter a command more than twice, put it in a script.</p>
<p>I don't remember who said this, but it is a good rule of thumb
for administrators and essentially everybody working on a shell.
In fact, that is one of the points, why I love working with Linux.</p>
<p>I found myself grep'ing again and again in the same files,
so I optimized the workflow.
Both scripts are in my <a href="https://github.com/georgwassen/Scripts">Scripts repository</a>
on Github.</p>
<h1 id="bibgrep">bibgrep</h1>
<p><code>bibgrep</code> started as a tool just to search in my BibLaTeX files.
But as grep usually only displays the matching line (or a constant number
of surrounding lines), I ended up repeatedly opening the found position(s)
in an editor.
Therefore, <code>bibgrep</code> uses <code>sed</code> to display whole entries up to the closing
brace (that sits always on its own line in my bib files).
With some <a href="http://www.wassen.net/colorful-prompts.html">coloring</a>, the output is better readable.</p>
<p>The most recent improvement is storing the identified BIB key in
the <a href="http://www.wassen.net/x11-clipboard.html">X11 clipboard</a> (using <code>xsel</code>).
This allows me to directly paste it in the editor or to use <code>texgrep</code>
subsequently.</p>
<h1 id="texgrep">texgrep</h1>
<p><code>texgrep</code> is hardly more than a one-liner,
but if no parameters are provided, it uses the X11 clipboard
as search term.
In my workflow, I use <code>bibgrep</code> to find an entry (or check if it exists)
and then use <code>texgrep</code> to find all places where it is cited:</p>
<div class="highlight"><pre><span class="nv">$ </span>bibgrep Kato
../../bib/linux.bib:120
@inproceedings<span class="o">{</span>Kato2010Airs,
<span class="o">[</span>...<span class="o">]</span>
<span class="o">}</span>
../../bib/linux.bib:344
@inproceedings<span class="o">{</span>Kato2008Modular,
<span class="o">[</span>...<span class="o">]</span>
<span class="o">}</span>
<span class="nv">$ </span>bibgrep Kato2010
../../bib/linux.bib:120
@inproceedings<span class="o">{</span>Kato2010Airs,
<span class="o">[</span>...<span class="o">]</span>
<span class="o">}</span>
<span class="nv">$ </span>texgrep
<span class="nv">TERM</span><span class="o">=</span><span class="s1">'Kato2010Airs'</span>
chap01b_related.tex:366:<span class="se">\c</span>itetitle<span class="o">{</span>Kato2010Airs<span class="o">}</span>
</pre></div>
<p>The first search for "Kato" found two entries, therefore the search
was repeated with the year "Kato2010".
This is unique and a following <code>texgrep</code> searches for the previously
identified key "Kato2010Airs"
and displays that this BIB entry is cited in one TEX file
(it displays also the line).</p>
<p>The next improvement would be to store this info and somehow direct
my vim instanct to jump to that line... </p>Vim Spell Checking2013-12-03T08:21:14+01:00Georg Wassentag:www.wassen.net,2013-12-03:vim-spell-checking.html<h1 id="build-in-spell-checker">Build-In Spell Checker</h1>
<p>Activate for the current buffer (optionally set the language):</p>
<div class="highlight"><pre>:setlocal spell
:setlocal spell spelllang=en_us
</pre></div>
<p>Source: <a href="http://robots.thoughtbot.com/vim-spell-checking/">Vim Spell-Checking</a> (there is help how to automatically turn spellchecking on for selected file types)</p>
<table>
<thead>
<tr>
<th>Command</th>
<th>Meaning</th>
</tr>
</thead>
<tbody>
<tr>
<td><code>]s</code></td>
<td>next marked word</td>
</tr>
<tr>
<td>`[s'</td>
<td>previous marked word</td>
</tr>
<tr>
<td><code>z=</code></td>
<td>show a list of proposed words</td>
</tr>
<tr>
<td><code>zg</code></td>
<td>add marked word to the list of correct words</td>
</tr>
<tr>
<td>'zG`</td>
<td>add marked word to a temporary list of correct words</td>
</tr>
</tbody>
</table>
<p>see also: <code>:help spell</code></p>
<h1 id="language-tool">Language Tool</h1>
<p>Java program for spell-checking (not tested, yet)</p>
<ul>
<li><a href="http://www.languagetool.org/usage/">Link</a></li>
<li><a href="http://www.vim.org/scripts/script.php?script_id=3223">Vim Plugin</a></li>
</ul>Vim Gems2013-12-03T08:14:02+01:00Georg Wassentag:www.wassen.net,2013-12-03:vim-gems.html<p>Small collections of tricky hacks.</p>
<h1 id="insert-date-and-timestamp">Insert date- and timestamp</h1>
<div class="highlight"><pre>:nnoremap <F5> "=strftime("%c")<CR>P
:inoremap <F5> <C-R>=strftime("%c")<CR>
:iab <expr> DTS strftime("%c")
</pre></div>
<p>The first line is for the Normal mode. It places the time stamp in a register and pastes it
at the cursor position when pressing <code>F5</code>.
The second line is for the Insert mode (again on hitting <code>F5</code>)
and the third line generates an expression
that replaces the characters "DTS" to a timestamp.
The selection of that expression is probably not the best idea,
because while typing this text, I got the timestamp instead of the three letters D, T, S.</p>
<p>Source: <a href="http://vim.wikia.com/wiki/Insert_current_date_or_time">Insert current date or time</a></p>Vim Configuration and Settings2013-12-03T08:11:45+01:00Georg Wassentag:www.wassen.net,2013-12-03:vim-configuration-and-settings.html<p>So far, this is a collection of bits and pieces about configuring Vim.</p>
<h1 id="basics">Basics</h1>
<p>The options can be set and read like variables with the ex command <code>:set</code>.
A single <code>:set</code> shows all options that differ from their default value.
An actual setting can be inspected with <code>:set {option}?</code>.
There are toggle options, string and number values.
Toggle options are set with <code>:set {toggle}</code> and reset with <code>:set no{toggle}</code>.
The other options are set with <code>:set {option}={value}</code>.
Options that are a list of flag characters can be modified
by adding or removing dedicated flags with <code>:set {flags}+=a</code> or <code>:set {flags}-=b</code>
(to avoid overwriting the whole set of flags).
Many other operations are possible (e.g. inverting a toggle option),
refer to the help page.
With <code>:verbose set {option}?</code>, vim displays where an option was set.</p>
<p>There are global settings and local settings only for the current buffer</p>
<div class="highlight"><pre>:set
:setlocal
:setglobal
</pre></div>
<p>See: <code>:help options</code></p>
<h1 id="config-files">Config files</h1>
<p>The main config file is <code>$HOME/.vimrc</code>.
If the option <code>exrc</code> is set, vim also reads an existing file <code>.vimrc</code> in the working directory.
It can be used to override some settings on a per-project base.</p>
<p>See: <code>:help vimrc</code></p>
<h1 id="modelines">Modelines</h1>
<p>Modelines are per-file settings. By default, vim checks the first and last 5 lines
for a modeline. It usually begins with a comment sign (depending on the programming
language e.g. <code>//</code> for C, <code>#</code> for Bash and Python, <code>%</code> for LaTeX).
Then follows the marker <code>vim:</code>.
Only the <code>set</code> command is supported to avoid hostile text files manipulating your
editor and system.</p>
<p>Two forms are supported:</p>
<div class="highlight"><pre>// vim: sw=3 ts=6
/* vim: set sw=3 ts=6 : */
</pre></div>
<p>The first form lists only the options to set separated either by space or colon.
The second form begins with <code>set</code> and ends with a colon that can be followed by
other text, for example a comment terminator.</p>
<p>See: <code>:help modeline</code></p>
<h1 id="further-reading">Further reading</h1>
<p>Some packages with default configurations or to manage the config file
will be described in an article about <a href="http://www.wassen.net/drafts/vim-plugins.html">plugins</a>.</p>Bash Debugger2013-11-22T07:45:00+01:00Georg Wassentag:www.wassen.net,2013-11-22:bash-debugger.html<h1 id="build-in-debugging-features">Build-in Debugging Features</h1>
<p>The simplest way is to activate debugging output with the <code>-x</code> command line option
or in the script with <code>set -x</code>.
This will print every line to <code>stderr</code> before it is executed.
In those lines, the variables are already replaced with their values,
therefore it's easier to see what's happening.
Combined with the option <code>-v</code> that prints each line as it was read from the script,
a malfunction can often be traced back to the line where it happened
and the variables can be analyzed.</p>
<p>The fourth <a href="http://www.wassen.net/prompts.html">prompt</a> <code>PS4</code> allows to set the marker for <code>-x</code> tracing lines:</p>
<div class="highlight"><pre><span class="nb">export </span><span class="nv">PS4</span><span class="o">=</span><span class="s1">'+${BASH_SOURCE}:${LINENO}:${FUNCNAME[0]}: '</span>
</pre></div>
<p>or, extended with <a href="http://www.wassen.net/colorful-prompts.html">colors</a>:</p>
<div class="highlight"><pre><span class="nb">export </span><span class="nv">PS4</span><span class="o">=</span><span class="s1">'+\[\033[1;33m\]${BASH_SOURCE}\[\033[0m\]:\[\033[0;31m\]${LINENO}\[\033[0m\]:\[\033[1;34m\]${FUNCNAME[0]}\[\033[0m\]: '</span>
</pre></div>
<p>If you're debugging a larger script,
you can activate tracing for only parts of the script:</p>
<div class="highlight"><pre><span class="nv">N</span><span class="o">=</span>4
<span class="nb">set</span> -x <span class="c"># start tracing</span>
<span class="nv">N</span><span class="o">=</span><span class="k">$((</span> N <span class="o">*</span> <span class="m">2</span> <span class="k">))</span>
<span class="nb">set</span> +x <span class="c"># stop tracing</span>
<span class="nb">echo</span> <span class="nv">$N</span>
</pre></div>
<h1 id="bashdb">BashDB</h1>
<p>The <a href="http://bashdb.sourceforge.net/">BashDB</a> is a wrapper that executes shell
scripts with debugging features such as breakpoints, inspecting variables
and changing the script under test.</p>
<h1 id="see-also">See also</h1>
<ul>
<li><a href="http://binfalse.de/2012/09/howto-debug-bash-scripts/">binfalse: HowTo Debug Bash Scripts</a></li>
<li><a href="http://stackoverflow.com/questions/4640794/a-running-bash-script-is-hung-somewhere-can-i-find-out-what-line-it-is-on">Examine running script</a></li>
<li><a href="http://8bitsof.me/bash-attack-part-1.html">Using GDB on bash running a script</a></li>
</ul>Colorful Prompts2013-11-22T07:45:00+01:00Georg Wassentag:www.wassen.net,2013-11-22:colorful-prompts.html<p>Color can be set in the shell with control sequences starting with <code>\033</code> and ending with <code>m</code>.
The <code>echo</code> command must be given the parameter <code>-e</code> to interpret the control sequences:</p>
<div class="highlight"><pre><span class="nb">echo</span> -e <span class="s1">'Hello \033[1;31mworld\033[0m!'</span>
</pre></div>
<p>The 033 is octal for 27 (decimal) which is the ASCII code for escape. </p>
<h2 id="text-color">Text color</h2>
<table>
<thead>
<tr>
<th>Sequence</th>
<th>Color</th>
</tr>
</thead>
<tbody>
<tr>
<td><code>\033[0;30m</code></td>
<td>black</td>
</tr>
<tr>
<td><code>\033[1;30m</code></td>
<td>dark gray</td>
</tr>
<tr>
<td><code>\033[0;31m</code></td>
<td>red</td>
</tr>
<tr>
<td><code>\033[1;31m</code></td>
<td>light red</td>
</tr>
<tr>
<td><code>\033[0;32m</code></td>
<td>green</td>
</tr>
<tr>
<td><code>\033[1;32m</code></td>
<td>light green</td>
</tr>
<tr>
<td><code>\033[0;33m</code></td>
<td>brown</td>
</tr>
<tr>
<td><code>\033[1;33m</code></td>
<td>yellow</td>
</tr>
<tr>
<td><code>\033[0;34m</code></td>
<td>blue</td>
</tr>
<tr>
<td><code>\033[1;34m</code></td>
<td>light blue</td>
</tr>
<tr>
<td><code>\033[0;35m</code></td>
<td>violet</td>
</tr>
<tr>
<td><code>\033[1;35m</code></td>
<td>light violet</td>
</tr>
<tr>
<td><code>\033[0;36m</code></td>
<td>teal</td>
</tr>
<tr>
<td><code>\033[1;36m</code></td>
<td>cyan</td>
</tr>
<tr>
<td><code>\033[0;37m</code></td>
<td>silver</td>
</tr>
<tr>
<td><code>\033[1;37m</code></td>
<td>white</td>
</tr>
<tr>
<td><code>\033[0m</code></td>
<td>default</td>
</tr>
</tbody>
</table>
<h2 id="background-color">Background color</h2>
<table>
<thead>
<tr>
<th>Sequence</th>
<th>Color</th>
</tr>
</thead>
<tbody>
<tr>
<td><code>\033[0m</code></td>
<td>default (no background)</td>
</tr>
<tr>
<td><code>\033[40;m</code></td>
<td>black background</td>
</tr>
<tr>
<td><code>\033[41;m</code></td>
<td>red background</td>
</tr>
<tr>
<td><code>\033[42;m</code></td>
<td>green background</td>
</tr>
<tr>
<td><code>\033[43;m</code></td>
<td>brown background</td>
</tr>
<tr>
<td><code>\033[44;m</code></td>
<td>blue background</td>
</tr>
<tr>
<td><code>\033[45;m</code></td>
<td>violet background</td>
</tr>
<tr>
<td><code>\033[46;m</code></td>
<td>teal background</td>
</tr>
<tr>
<td><code>\033[47;m</code></td>
<td>silver background</td>
</tr>
</tbody>
</table>
<h2 id="example-script">Example script</h2>
<div class="highlight"><pre><span class="k">for</span> FG in <span class="m">30</span> <span class="m">31</span> <span class="m">32</span> <span class="m">33</span> <span class="m">34</span> <span class="m">35</span> <span class="m">35</span> <span class="m">36</span> 37<span class="p">;</span> <span class="k">do</span>
<span class="k">for</span> LI in <span class="m">0</span> 1<span class="p">;</span> <span class="k">do</span>
<span class="nb">echo</span> -en <span class="s2">"\033[</span><span class="si">${</span><span class="nv">LI</span><span class="si">}</span><span class="s2">;</span><span class="si">${</span><span class="nv">FG</span><span class="si">}</span><span class="s2">m</span><span class="si">${</span><span class="nv">LI</span><span class="si">}</span><span class="s2">;</span><span class="si">${</span><span class="nv">FG</span><span class="si">}</span><span class="s2"> "</span>
<span class="k">for</span> BG in <span class="m">40</span> <span class="m">41</span> <span class="m">42</span> <span class="m">43</span> <span class="m">44</span> <span class="m">45</span> <span class="m">46</span> 47<span class="p">;</span> <span class="k">do</span>
<span class="nb">echo</span> -en <span class="s2">"\033[</span><span class="si">${</span><span class="nv">BG</span><span class="si">}</span><span class="s2">m </span><span class="si">${</span><span class="nv">BG</span><span class="si">}</span><span class="s2"> "</span>
<span class="k">done</span>
<span class="nb">echo</span> -e <span class="s2">"\033[0m"</span>
<span class="k">done</span>
<span class="k">done</span>
</pre></div>
<p>Text and background color can be set in one control sequence:</p>
<div class="highlight"><pre><span class="nb">echo</span> -e <span class="s2">"Print \033[44;1;33myellow on blue\033[0m"</span>
</pre></div>
<h1 id="prompts">Prompts</h1>
<p>To use colors in <a href="http://www.wassen.net/prompts.html">prompts</a>, just use the same codes,
but enclose them in <code>\[</code> and <code>\]</code>.
These markers are required to avoid misinterpretations
-- they start and end a sequence of non-printable control sequences --
probably for shells not supporting the color settings.</p>
<div class="highlight"><pre><span class="nb">export </span><span class="nv">PS1</span><span class="o">=</span><span class="s1">'\[\033[1;34m\]\u\[\033[0m\]@\[\033[0;32m\]\h\[\033[0m\]:\[\033[1;33m\]\w\[\033[0m\]$ '</span>
</pre></div>
<p>This example changes the color to light blue, prints the username <code>\u</code>,
sets the color back to default, print the character "@",
changes to green, prints the hostname <code>\h</code>,
then a colon in the default color, the working directory <code>\w</code> in yellow
and a dollar sign in the default color.</p>
<p>Valuable sources:</p>
<ul>
<li><a href="https://wiki.archlinux.org/index.php/Color_Bash_Prompt">archlinux: Color Bash Prompt</a></li>
</ul>Prompts2013-11-22T07:45:00+01:00Georg Wassentag:www.wassen.net,2013-11-22:prompts.html<p>The Bash supports four different prompts:</p>
<ul>
<li><code>PS1</code> is the usual prompt when interactively entering commands</li>
<li><code>PS2</code> is used, when a command spans multiple lines
(e.g. if a string is still open when you press ENTER or after a Backslash)</li>
<li><code>PS3</code> is used for shell menues with <code>select</code></li>
<li><code>PS4</code> is prepended for execution traces (see <a href="http://www.wassen.net/bash-debugger.html">Bash Debugging</a>)</li>
</ul>
<p>The Prompts <code>PS1</code>, <code>PS2</code> and <code>PS4</code> support placeholders that are replaced with current information
such as the hostname, the username or the working directory
(the full list is in the Bash man page and documentation).</p>
<table>
<thead>
<tr>
<th>Symbol</th>
<th>Description</th>
</tr>
</thead>
<tbody>
<tr>
<td><code>\h</code></td>
<td>Hostname</td>
</tr>
<tr>
<td><code>\u</code></td>
<td>User name</td>
</tr>
<tr>
<td><code>\w</code></td>
<td>Current working directory</td>
</tr>
<tr>
<td><code>\@</code></td>
<td>The character "@"</td>
</tr>
<tr>
<td><code>\$</code></td>
<td>For root "#", otherwise "$"</td>
</tr>
</tbody>
</table>
<p>An example prompt:</p>
<div class="highlight"><pre><span class="nv">PS1</span><span class="o">=</span><span class="s1">'\u@\h:\w$ '</span>
</pre></div>
<p>This prints the username, "@" hostname, a colon, the working directory and a dollar sign (followed by a space).</p>
<p>Example for a <a href="http://www.wassen.net/colorful-prompts.html">colored</a> prompt:</p>
<div class="highlight"><pre><span class="nb">export </span><span class="nv">PS1</span><span class="o">=</span><span class="s1">'\[\033[1;34m\]\u\[\033[0m\]@\[\033[0;32m\]\h\[\033[0m\]:\[\033[1;33m\]\w\[\033[0m\]$ '</span>
</pre></div>Encoding2013-11-20T10:25:00+01:00Georg Wassentag:www.wassen.net,2013-11-20:encoding.html<p>Today, I was asked for help with encoding problems with <code>vim</code>.
Since I know that my colleage uses Windows on his desktop computer and uses a SSH connection
to a Linux box, we first made sure, that the shell-terminal connection worked properly.</p>
<p>Files and network connections are a byte stream.
But both the terminal and the shell use an internal string representation.
With unicode, a single character can be made up of one to four bytes.</p>
<h1 id="shell-terminal">Shell - Terminal</h1>
<p>In the shell, the encoding can be set with the environment variable <code>LANG</code>
(or variants of <code>LC_*</code>).
On my German language system, I use <code>LANG=de_DE.UTF-8</code>.
Both the shell and the SSH client must be configured similarly.</p>
<p>If they mismatch, funny things can be watched.
When typing an <em>Umlaut</em> (some special character that is not in basic ASCII, like the German "ö"),
it appears on the screen, but removing it requires typing backspace two times.
Or the <em>Umlaut</em> does not appear immediately, but after another key is pressed.
This can be explained with the conversion of characters to a byte stream
both for sending the entered key to the shell and for receiving the new sceen content.</p>
<p>If the shell uses <em>latin1</em> and terminal is configured to <em>UTF-8</em>,
the <em>Umlaut</em> is send as two byte code to the shell
which interprets the two bytes as two strange symbols and displays them.
But the terminal interprets those two bytes again as <em>Umlaut</em>.
So far, everything looks fine.
But when backspace is pressed, the shell removes one of the stange symbols
and the terminal misses the second byte of the two-byte <em>Umlaut</em> code.
Only after a second press of Backspace, all Bytes of the <em>Umlaut</em> are gone
and the character string is clean again.</p>
<p>In the opposite case, if the shell uses <em>utf-8</em> and the SSH client uses <em>latin1</em>,
the <em>Umlaut</em> is sent as a number between 128 and 255 over the network.
Those numbers alone are invalid UTF-8 codes because they indicate a multi-byte character.
After another key press on the terminal, the additional byte can complete the UTF-8 sequence
(but it may happen that it requires additional bytes).
For the shell, this is a two-byte character (and it can only be removed together),
but the SSH client will happily display the two original <em>latin1</em> characters.</p>
<h1 id="applications">Applications</h1>
<p>The next step is to ensure, that the application works correctly together
with the shell.
I made the experience, that vim usually works correctly this way.
But it may misinterpret the file format.
Again, vim reads a stream of bytes from a file and manages them as characters.
Those are printed to the screen with the encoding that the shell uses.</p>
<p>In <em>vim</em>, the setting <code>:set fileencoding</code> can be used to change how a file
is interpreted.
To be continued...</p>Git2013-11-14T16:11:00+01:00Georg Wassentag:www.wassen.net,2013-11-14:git.html<p><a href="http://git-scm.com/">Git</a> is the distributed version control system
that was developed for the Linux kernel
(started by Linux Torvalds in 2005).
Today, it is widely used in the open source development community.</p>
<p>At work, I got in contact with Subversion and I really enjoy using it for all text-based projects
(programming, LaTeX, you name it).
However, the Linux kernel is managed with git
and in recent time, many projects switched to Git.
So I thought it was time to learn its basics.
And while learning, I maintain my <em>cheat sheet</em> here.
It is not intended to be a stand-alone tutorial,
there are many good ones available
(e.g. the <a href="http://git-scm.com/book">Book</a> on the Git homepage,
that was also translated in a number of languages).
So this is merely a short reference.
Further, my own view is from an experienced Suversion user,
so I base my notes on previous knowledge
about source code revision systems.</p>
<p>While Subversion is a server-based version control system
(the central repository is a server and all interaction is done with this server),
Git is a <em>distributed</em> version control system.
That means that every user has a local repository (therefore "distributed")
that stores all revisions.
These local repos can be synchronized by pushing the changes to a server
or by pulling changes from someone into the local repo.</p>
<p>A git server can be any system accessible via network either with SSH or HTTP(s).
<a href="https://github.com">Github</a> provides free Git hosting.
For the following tests, I use my
<a href="https://github.com/georgwassen/HelloWorld">test repository</a>.</p>
<h1 id="configuration">Configuration</h1>
<p>Every tutorial starts with the following lines to configure your name and E-Mail:</p>
<div class="highlight"><pre><span class="nv">$ </span>git config --global user.name <span class="s2">"Your Name Comes Here"</span>
<span class="nv">$ </span>git config --global user.email you@yourdomain.example.com
</pre></div>
<p>So it appears to be really advisable to do this.
The background is that Git stores the name and an E-Mail address with every commit.
Unlike Subversion, where every user needs an account on the server,
Git cryptographically hashes all commits and everybody (or every alias)
can create commits.
The <code>--global</code> parameter adds the settings to the config file in your home directory.
If you want to contribute to a project with a different identity
(e.g. your company E-Mail instead of your private one),
the settings can also be done only for the current project by omitting this option.
The global config file is <code>~/.gitconfig</code> and the project-one is <code>.git/config</code>.</p>
<p>Please refer to the documentation (man git-config and <code>git --help</code>) for further information.
The command <code>git config -l</code> lists the current settings
(if multiple entries exist, the last one overwrites previous ones).</p>
<p>Activate colors (which are deactivated by default) with:</p>
<div class="highlight"><pre><span class="nv">$ </span>git config --global color.diff auto
<span class="nv">$ </span>git config --global color.status auto
<span class="nv">$ </span>git config --global color.branch auto
</pre></div>
<h1 id="initialization">Initialization</h1>
<p>A new repository is initialized by entering</p>
<div class="highlight"><pre><span class="nv">$ </span>git init
</pre></div>
<p>in the base directory. If that's an existing project,
you can add all files with</p>
<div class="highlight"><pre><span class="nv">$ </span>git add *
<span class="nv">$ </span>git commit -m <span class="s2">"initial commit"</span>
</pre></div>
<p>Instead of <code>git add *</code>, you want probably add only selected files, such as <code>*.c</code> etc.</p>
<p>It's also possible to <em>clone</em> an existing repository (like a Subversion checkout):</p>
<div class="highlight"><pre><span class="nv">$ </span>git clone https://github.com/georgwassen/HelloWorld
</pre></div>
<p>The URL is provided by the project that offers the repository.</p>
<h1 id="first-steps">First steps</h1>
<p>Files, that should be commited to the repository must be <em>added</em> (called "staging"):</p>
<div class="highlight"><pre><span class="nv">$ </span>git add main.c
<span class="nv">$ </span>git add Makefile
</pre></div>
<p>Check the changes:</p>
<div class="highlight"><pre><span class="nv">$ </span>git status
<span class="nv">$ </span>git show
</pre></div>
<p>Commit the staged changes (to the local repository) providing a change-log message:</p>
<div class="highlight"><pre><span class="nv">$ </span>git commit -m <span class="s2">"initially adding my files"</span>
</pre></div>
<p>If you don't provide the <code>-m "message"</code> parameter, Git will open an editor
and ask for the commit message.</p>
<p>Now, further modifications can be done.
Unlike Subversion, registered files are not automatically included in the next commit.
With Git, the files must be added again:</p>
<div class="highlight"><pre><span class="nv">$ </span>git status
<span class="c"># On branch master</span>
<span class="c"># Changes not staged for commit:</span>
<span class="c"># (use "git add <file>..." to update what will be committed)</span>
<span class="c"># (use "git checkout -- <file>..." to discard changes in working directory)</span>
<span class="c">#</span>
<span class="c"># modified: main.c</span>
<span class="c">#</span>
<span class="c"># Untracked files:</span>
<span class="c"># (use "git add <file>..." to include in what will be committed)</span>
<span class="c">#</span>
<span class="c"># hello</span>
<span class="c"># main.o</span>
no changes added to commit <span class="o">(</span>use <span class="s2">"git add"</span> and/or <span class="s2">"git commit -a"</span><span class="o">)</span>
<span class="nv">$ </span>git add main.c
<span class="nv">$ </span>git status
<span class="c"># On branch master</span>
<span class="c"># Changes to be committed:</span>
<span class="c"># (use "git reset HEAD <file>..." to unstage)</span>
<span class="c">#</span>
<span class="c"># modified: main.c</span>
<span class="c">#</span>
<span class="c"># Untracked files:</span>
<span class="c"># (use "git add <file>..." to include in what will be committed)</span>
<span class="c">#</span>
<span class="c"># hello</span>
<span class="c"># main.o</span>
<span class="nv">$ </span>git commit -m <span class="s2">"added Bye message"</span>
<span class="o">[</span>master 6c8bb69<span class="o">]</span> added Bye message
<span class="m">1</span> file changed, <span class="m">1</span> insertion<span class="o">(</span>+<span class="o">)</span>
</pre></div>
<p>The cycle is:</p>
<ul>
<li>Make (and test) changes. Prefer small and related changes and commit them with meaningful messages.</li>
<li>Add changed files: <code>git add file.c</code></li>
<li>Check state and if you missed a file: <code>git status</code></li>
<li>Commit changes to local repository: <code>git commit -m "message"</code></li>
</ul>
<p>So far, all work is done in the <em>master</em> branch.</p>
<h1 id="upload-to-a-server">Upload to a server</h1>
<p>As explained in the <a href="https://help.github.com/articles/create-a-repo">Github help</a>,
a local repository can be pushed to a server with the command</p>
<div class="highlight"><pre><span class="nv">$ </span>git remote add origin git@github.com:georgwassen/HelloWorld.git
<span class="nv">$ </span>git push origin master
</pre></div>
<p>(Using the URL for my test repos with SSH access.)</p>
<p>Later, or if the local repository was <em>cloned</em> from a server,
it suffices to issue:</p>
<div class="highlight"><pre><span class="nv">$ </span>git push origin master
</pre></div>
<p>(`origin´ is the name of the server and ´master´ is the branch to push)</p>
<h1 id="the-full-power-of-git">The full power of git</h1>
<p>It is possible to synchronize the local repository with multiple servers.</p>
<p>Get the changes from a server:</p>
<div class="highlight"><pre><span class="nv">$ </span>git remote add upstream https://github.com/octocat/Spoon-Knife.git
<span class="nv">$ </span>git fetch upstream
<span class="nv">$ </span>git merge upstream/master
</pre></div>
<p>The strongest feature of Git is said to be branching and merging.
Create a new branch and switch to it (there should be no open changes, i.e. a clean working copy):</p>
<div class="highlight"><pre><span class="nv">$ </span>git branch newfeature
<span class="nv">$ </span>git checkout newfeature
</pre></div>
<p>With <code>git branch</code>, all availabel branches are displayed.
Now, you can switch between the branches and commit changes.</p>
<p>To merge changes from one branch to the other (for example, to merge
the changes developed in a branch back to master), just call</p>
<div class="highlight"><pre><span class="nv">$ </span>git checkout master
<span class="nv">$ </span>git merge newfeature
</pre></div>
<p>A merged branch can be removed with <code>git branch -d newfeature</code>.</p>
<p>Branches in Git are very light-weight and fast,
so they can be used to keep separate issues apart
and merge them when they work.
If there is a merge conflict (e.g. if both branches changed the same line),
they conflict is marked and reported.
You need to clean up the conflicts manually
and then <code>add</code> the conflicting files to the staging area.
When all conflicts are resolved, commit the staged files
and the pending merge will be completed.</p>
<p>To get an overview of the current branch and how it's composed of commits
and merges, the visual tool <code>gitk</code> is a great help.
If you start it out of a Git repository, it displays the log of the
current branch.
With <code>gitk --all</code>, it displays all existing branches which helps to remember
what branches are pending for a merge.</p>
<h1 id="creating-a-server-repository">Creating a server repository</h1>
<p>Git supports four protocols to interact with remote repositories:
* <em>File</em>: for local repos in other directories or network drives (e.g. NFS)
* <em>SSH</em>: encrypted transfer for retrieving <em>and</em> uploading
* <em>HTTP/HTTPS</em>: only for download (easy publishing of source code)
* <em>GIT</em>: only for download</p>
<p>Refer to the documentation for benefits and disadvantages
and for details how to set these protocols up.</p>
<p>To create a personal repository on a Linux server accessible with SSH on the internet,
I followed these steps:</p>
<ol>
<li>
<p>Create a user dedicated for Git (or use your personal account).
The commits are already tagged with name and E-Mail,
the <code>git</code> user is used to determine read-only or read/write access on the repo.
I copied the <code>~/.ssh/authorized_users</code> from another account to enable private/public-key-logins.</p>
<div class="highlight"><pre><span class="nv">$ </span>ssh www.example.com
<span class="nv">$ </span>su -
<span class="c"># adduser -m git</span>
<span class="c"># su git</span>
<span class="nv">$ </span>mkdir .ssh
<span class="nv">$ </span>cp authorized_keys ~/.ssh
</pre></div>
<p>Now, I can login to the server with <code>ssh git@www.example.com</code> without being asked for a password.
Note, that every user that should access the Git repository via SSH protocol,
must provice a SSH public key and subsequently can also log-in on the server!
(The <code>git</code> user can be configured to have no shell to disallow logging in,
but that's out of the scope of this article.)</p>
</li>
<li>
<p>Create a Git repository.
Usually, a server repo should be a <em>bare</em> one where no working copy exists.
On my server, an old version of Git does not know the parameter, but this is how it should work:</p>
<div class="highlight"><pre><span class="nv">$ </span>mkdir -p git/HelloWorld.git
<span class="nv">$ </span><span class="nb">cd </span>git/HelloWorld.git
<span class="nv">$ </span>git init --bare
</pre></div>
<p>Now, the new repository has the Git URL <code>git@www.example.com:git/HelloWorld.git</code>.
It is a convention to name bare repositories with a <code>.git</code> extension.
When cloning such a repository, the local directory is named without
this extension by default.</p>
</li>
<li>
<p>Add the new upstream repository to your local Git repository (where the working copy lives):</p>
<div class="highlight"><pre><span class="nv">$ </span>git remote add myrepo git@www.example.com:git/HelloWorld.git
<span class="nv">$ </span>git push myrepo master
</pre></div>
<p>With <code>git branch -a</code> or <code>gitk</code>, you can see, that two remote repositories are listed now.</p>
</li>
</ol>
<h1 id="tips-and-tricks">Tips and Tricks</h1>
<h2 id="amend-to-a-commit">Amend to a commit</h2>
<p>If you see the typo in the commit message shortly after hitting ENTER
or forgot to compile the change and committed an error,
the last commit can be updated or <em>amended</em>.
Use <code>git commit --amend</code> to edit the last commit message
or add and commit an additional file (or a file again) to add that
change to the last commit.</p>
<h2 id="backdated-branch">Backdated Branch</h2>
<p>Ever started commiting changes and then determined that this should better have gone into a branch?
It's easy to move the last N commits into a new branch:</p>
<ol>
<li>create the new branch (it contains all commits in the current state,
thus also the ones that should be moved into it).</li>
<li>reset the current branch (not the new one) to remove the additional commits.</li>
<li>checkout the new branch and continue.<div class="highlight"><pre><span class="nv">$ </span>git branch newbranch
<span class="nv">$ </span>git reset --hard HEAD~3
<span class="nv">$ </span>git checkout newbranch
</pre></div>
</li>
</ol>
<p>Again: the <code>reset</code> is done on the master (or previous) branch where the last changes should be removed.
The commits are preserved in the new branch.</p>
<p>Source: <a href="http://stackoverflow.com/questions/1628563/how-can-i-move-recent-commits-to-a-new-branch-with-git">Stackoverflow</a></p>
<h2 id="stashing">Stashing</h2>
<p>When changing between branches, the repository should be in a clean state.
If you have modifications that are not yet ready for a commit
but you need to change to another branch (e.g. for a hot fix),
you can <em>stash</em> the pending changes.
That's similar to a commit, but temporary.</p>
<div class="highlight"><pre><span class="nv">$ </span>git stash <span class="o">[</span>save<span class="o">]</span>
<span class="nv">$ </span>git checkout otherbranch
<span class="c"># do some modifications, e.g. to fix an error</span>
<span class="nv">$ </span>git commit -am <span class="s2">"hotfix..."</span>
<span class="nv">$ </span>git checkout firstbranch
<span class="nv">$ </span>git stash pop
</pre></div>
<p>The stashed changes can be applied to another branch
or on the branch where you stashed them.
The stashing mechanism uses a stack of stashes.
The command <code>git stash list</code> shows the stack of stashes
and every line gives hint in which branch it was created.
See <a href="http://git-scm.com/book/en/Git-Tools-Stashing">the book</a> and the man-page <code>git-stash</code>
for more details.</p>
<h2 id="import-from-subversion">Import from Subversion</h2>
<p>This part was moved to a dedicated <a href="http://www.wassen.net/git-subversion.html">article</a>.</p>
<h2 id="display-current-branch-in-the-bash-prompt">Display current branch in the Bash prompt</h2>
<p>The bash-completion package contains a function that supports the display of
the current Git branch in the <a href="http://www.wassen.net/prompts.html">Bash prompt</a>.</p>
<ol>
<li>
<p>Check, if the function <code>__git_ps1</code> exists:</p>
<div class="highlight"><pre><span class="nv">$ </span><span class="nb">type </span>__git_ps1
</pre></div>
<p>If it displays a long function, proceed with step 2, otherwise, try the following:</p>
<ul>
<li>Install with the package manager, look for packages like <code>bash-completion</code>, <code>git-extras</code> etc.</li>
<li>On Fedora, I found the file <code>/usr/share/git-core/contrib/completion/git-prompt.sh</code>
and copied it to <code>etc/bash_completion.d/</code>.</li>
<li>Google for that function and add it either to your local <code>.bash_profile</code>
or to the global <code>/etc/bash_completion.d</code>.</li>
</ul>
</li>
<li>
<p>Include <code>$(__git_ps1)</code> in your <code>PS1</code> definition. Example (with <a href="http://www.wassen.net/colorful-prompts.html">Colors</a>):</p>
<div class="highlight"><pre><span class="nb">export </span><span class="nv">PS1</span><span class="o">=</span><span class="s1">'\[\033[01;32m\]\u@\h\[\033[01;34m\] \w\[\033[01;33m\]$(__git_ps1)\[\033[01;34m\] \$\[\033[00m\] '</span>
</pre></div>
</li>
</ol>
<h2 id="push-into-working-repository">Push into Working Repository</h2>
<p>Usually, one should only push into a bare server repository.
But sometimes, it happens that I clone a working repository from one PC (e.g. to my Notebook)
and later push the changes back to continue working on the PC.
When pushing to a working repository, the <code>.git</code> is updated, but the checked out branch
is not modified (it's a bit like the difference between <code>fetch</code> and <code>pull</code>).</p>
<p>Now, the checked out working copy is out of sync with the <code>.git</code> repository backend data.
This can be fixed with the simple command:</p>
<div class="highlight"><pre>git checkout -f HEAD
</pre></div>
<p><strong>Warning:</strong> the current branch should be clean because that command will overwrite modifications.</p>
<h1 id="references">References</h1>
<p>I try to continue this little cheat sheet.
But there's already a load of great tutorials on the 'net.</p>
<ul>
<li>Free Book: <a href="http://git-scm.com/book">Pro Git</a></li>
<li><a href="http://www.vogella.com/articles/Git/article.html">Git Tutorial</a></li>
<li>Crash Course for Subversion Users <a href="http://git.or.cz/course/svn.html">Git - SVN Crash Course</a></li>
<li>Video Tutorials<ul>
<li><a href="https://www.youtube.com/watch?v=U8GBXvdmHT4">Github Webcast: The Basics of Git and GitHub</a> (51m)
Good first intro with history, background and basic usage.</li>
<li><a href="https://www.youtube.com/watch?v=ZDR433b0HJY">Scott Chacon: Introduction to Git</a> (1h 22m)
Introduction: He talks very fast and one should already have understood the basics.
But then it is a very valuable overview of the power of Git.</li>
<li><a href="https://www.youtube.com/watch?v=MYP56QJpDr4">Tim Berglund: Git From the Bits Up</a> (55m)
Advanced look at Git </li>
<li><a href="https://www.youtube.com/watch?v=ig5E8CcdM9g">Matthew McCullogh: Advanced Git</a> (1h 08m)
Explaining the inner workings of Git (plumbing layer).</li>
</ul>
</li>
</ul>Overview2013-10-29T20:07:00+01:00Georg Wassentag:www.wassen.net,2013-10-29:overview.html<p>These are books (some free on the web, some not), that
every programmer and computer engineer should know (at least).
I try to keep most of them in my L1 cache (i.e. on my desk).</p>
<h1 id="hardware">Hardware</h1>
<ul>
<li>
<p>Patterson, Hennessy: "Computer Organization and Design"</p>
</li>
<li>
<p>Hennessy, Patterson: "Computer Architecuture: A Quantitative Approach"</p>
</li>
</ul>
<h2 id="x86-architecture">x86 Architecture</h2>
<h3 id="intel-64-and-ia-32-architectures-software-developer-manuals">Intel 64 and IA-32 Architectures Software Developer Manuals</h3>
<p>The reference for programming Intel's x86 processors on the operating system level.</p>
<ul>
<li>Volume 1: Basic Architecture (important foundation)</li>
<li>Volume 2 (a, b): Instruction Set Reference (explains all assembly instructions in detail)</li>
<li>Volume 3 (a, b, c): System Programming (details about subsystems and features, such as cache, interrupts, power management, etc.)</li>
</ul>
<p>Download <a href="http://www.intel.com/content/www/us/en/processors/architectures-software-developer-manuals.html">link</a>
(if link is outdated, go to <a href="http://developer.intel.com">Intel's Website</a> and search for "Software Developer Manual").</p>
<h3 id="intel-64-and-ia-32-architectures-optimization-reference-manual">Intel 64 and IA-32 Architectures Optimization Reference Manual</h3>
<p>Many details about the microarchitectures (Nehalem, Sandy Bridge, Haswell) and their specific behavior.</p>
<p>Download <a href="http://www.intel.com/content/www/us/en/architecture-and-technology/64-ia-32-architectures-optimization-manual.html">link</a>
(if link is outdated, go to <a href="http://developer.intel.com">Intel's Website</a> and search for "Optimization Reference Manual").</p>
<h3 id="intel-chipsets">Intel Chipsets</h3>
<p>There are also manuals for Intel's chipsets (e.g. ICH 8/9/10),
where their configuration is described (e.g. the details on the I/O APIC).</p>
<h3 id="amd64-architecture-programmers-manual">AMD64 Architecture Programmer's Manual</h3>
<p>The reference for programming AMD's processors on the operating system level.</p>
<ul>
<li>Volume 1: Application programming</li>
<li>Volume 2: System Programming</li>
<li>Volume 3: General Purpose and System Instructions (Assembler Reference)</li>
<li>Volume 4: 128-bit and 256 bit media instructions (SSE and AVX Reference)</li>
<li>Volume 5: 64-Bit Media and x87 Floating-Point Instructions (MMX and FPU Reference)</li>
</ul>
<p>Download <a href="http://developer.amd.com/resources/documentation-articles/developer-guides-manuals/">link</a>
(if link is outdated, go to <a href="http://www.amd.com">AMD's Website</a> and search for "Developer Guides and Manuals").</p>
<p>Also on that page:</p>
<ul>
<li>Software Optimization Guide (with Details on specific processor versions)</li>
</ul>
<h2 id="low-level-details-for-optimization">Low-level Details for Optimization</h2>
<p>This selections is biased for x86 systems and the PC architecture
(which is also employed in most HPC systems).</p>
<ul>
<li>
<p>Hager: "Introduction to High Performance Computing for Scientists and Engineers"</p>
</li>
<li>
<p>Fog: <a href="http://www.agner.org/optimize/">"Software Optimization Manuals"</a></p>
</li>
<li>
<p>Drepper: "What every programmer should know about memory" <a href="http://www.akkadia.org/drepper/cpumemory.pdf">Link</a></p>
</li>
</ul>
<h1 id="programming-and-algorithms">Programming and Algorithms</h1>
<ul>
<li>Sedgewick</li>
</ul>
<h1 id="c-programming">C Programming</h1>
<ul>
<li>Kerningham & Richie: "The C Programming Language"</li>
</ul>
<h2 id="programming-tools">Programming Tools</h2>
<ul>
<li>
<p><a href="http://svnbook.red-bean.com/index.en.html">Version control with Subversion</a></p>
<ul>
<li>there are translations (e.g. <a href="http://svnbook.red-bean.com/index.de.html">German</a>),
but they are not up to date with the English version.</li>
</ul>
</li>
<li>
<p><a href="http://git-scm.com/book">Git Book</a> </p>
<p>thorough introduction and documentation to the distributed source code version control system <em>Git</em></p>
<ul>
<li>also in other languages, e.g. <a href="http://git-scm.com/book/de">German</a></li>
</ul>
</li>
</ul>
<h1 id="shell-programming-and-bash">Shell Programming and Bash</h1>
<ul>
<li>Powers, Peek, O'Reilly, and Loukides: "Unix Power Tools"</li>
</ul>
<h1 id="python">Python</h1>
<ul>
<li>...</li>
</ul>
<h1 id="operating-systems">Operating Systems</h1>
<p>General basics on operating systems:</p>
<ul>
<li>
<p>Tanenbaum: "Modern Operating Systems"</p>
</li>
<li>
<p>Stallings: "Operating Systems: Internals and Design Principles"</p>
</li>
<li>
<p>Silberschatz: "Operating System Concepts"</p>
</li>
</ul>
<h2 id="linux-kernel">Linux Kernel</h2>
<ul>
<li>
<p>Bovet: "Understanding the Linux Kernel"</p>
</li>
<li>
<p>Love: "Linux Kernel Development"</p>
</li>
<li>
<p>Corbet: "Linux Device Drivers"</p>
</li>
</ul>
<h2 id="linux-application-development">Linux Application Development</h2>
<ul>
<li>
<p>Kerrisk: "The Linux Programming Interface"</p>
</li>
<li>
<p>Rochkind: "Advanced UNIX Programming"</p>
</li>
</ul>
<h2 id="special-topics">Special topics</h2>
<ul>
<li>
<p>Nichols, Buttlar, and Proulx Farrell: "Pthreads Programming"</p>
</li>
<li>
<p>Gallmeister: "Posix.4: Programming for the real world"</p>
</li>
</ul>
<h2 id="parallel-systems-and-concurrent-programming">Parallel systems and concurrent programming</h2>
<ul>
<li>
<p>Herlihy: "The Art of Multiprocessor Programming"</p>
</li>
<li>
<p>Vajda: "Programming Many-Core Chips" (2011)</p>
</li>
</ul>
<h2 id="real-time-systems">Real-Time Systems</h2>
<ul>
<li>
<p>Liu: "Real-Time Systems"</p>
</li>
<li>
<p>Burns: "Real-Time Systems and Programming Languages"</p>
</li>
</ul>
<h3 id="multi-processors-in-real-time">Multi-processors in Real-Time</h3>
<ul>
<li>
<p>Moyer: "Multicore Embedded Systems" (2013)</p>
<ul>
<li>Very thorough textbook about multi-processor systems in embedded and real-time applications</li>
</ul>
</li>
<li>
<p>Domeika: "Software Development for Embedded Multi-core Systems" (2008)</p>
<ul>
<li>Subtitle: "A Practical Guide for Using Embedded Intel Architecture"</li>
<li>concentrates on the x86 architecture for embedded systems</li>
</ul>
</li>
</ul>
<h1 id="more-free-books">More free books</h1>
<p>Wikibooks: <a href="http://en.wikibooks.org/wiki/A_Little_C_Primer">"A Little C Primer"</a></p>
<p>Wikibooks: <a href="http://en.wikibooks.org/wiki/C_Programming">"C Programming"</a></p>
<p>Wikibooks: <a href="http://en.wikibooks.org/wiki/LaTeX">"LaTeX"</a></p>
<p>Wikibooks: <a href="http://en.wikibooks.org/wiki/X86_Disassembly">"x86 Disassembly"</a></p>Annotating2013-10-29T10:22:00+01:00Georg Wassentag:www.wassen.net,2013-10-29:annotating.html<p>Two LaTeX packages to add comments to documents while working together with others.</p>
<h1 id="package-todonotes">Package <code>todonotes</code></h1>
<p>The <code>todonotes</code> package allows inserting text boxes on the margin of the page.
They are linked to the position in the text, where the <code>\todo{}</code> is entered.</p>
<p>An <a href="https://www.writelatex.com/read/sbnmzgrfkwyp">example</a> can be seen at WriteLatex.</p>
<h1 id="package-trackchanges">Package <code>trackchanges</code></h1>
<p>This package displays notes at the page margins similar to the above mentioned.
However, it not just collects <em>to do's</em>, but supports a history of changes.</p>
<p>How this can be used is best seen in an <a href="https://www.writelatex.com/read/cymjcmjsnphy">example</a> at WriteLatex.</p>Quotes2013-10-18T10:12:00+02:00Georg Wassentag:www.wassen.net,2013-10-18:quotes.html<h1 id="31-october-2014">31 October 2014</h1>
<p>"In the majority of cases, performance will be programmer bound" - Barker's Law (as in Mike Barker)</p>
<ul>
<li>Nitsan Wakart, <a href="https://twitter.com/nitsanw/status/525963489400344576">Twitter</a>
via <a href="http://highscalability.com/blog/2014/10/31/stuff-the-internet-says-on-scalability-for-october-31st-2014.html">Highscalability.com</a></li>
</ul>
<h1 id="18-october-2013">18 October 2013</h1>
<blockquote>
<p>Single core systems are becoming a historic curiosity, we should justify every piece of extra complexity we add for them.</p>
</blockquote>
<ul>
<li>Ingo Molnar (via LWN <a href="https://lwn.net/Articles/570459/">Quote of the Week</a>)</li>
</ul>
<h1 id="10-januar-2013">10 Januar 2013</h1>
<blockquote>
<p>A programmer had a problem. He thought to himself, "I know, I'll solve it with threads!". has Now problems. two he</p>
</blockquote>
<ul>
<li>Davidlohr Bueso (via LWN <a href="http://lwn.net/Articles/531841/">Quote of the Week</a>)</li>
</ul>Problems2013-10-03T09:41:00+02:00Georg Wassentag:www.wassen.net,2013-10-03:problems.html<p>Will Haldean Brown wrote about <a href="http://haldean.org/docstore/?vim-problems">The problem with Vim</a>.
This made me smile, because I know the same situations.</p>
<p>But one thing bugs me even more:
Every vi user knows the repetitive task of opening a file, making some changes
and then: save and close, ESC, <code>:wq</code>, ENTER.
That sequence is so engraved in the muscles, we don't think about it.</p>
<p>But once in a while, after entering text in a GUI window,
it happens when the intended action of store and exit makes
the fingers hit the well known sequence of ESC, ...
well, the other keys remain unheard by the canceled dialog window.</p>
<p>Okay, there are plugins for many applications to emulate vi keybindings,
but there will be another dialog
that has ESC hard wired with the cancel button.</p>
<p>Still not resolved this...</p>Endianess2013-09-13T17:53:00+02:00Georg Wassentag:www.wassen.net,2013-09-13:endianess.html<p>For multi-byte variables as <code>int</code>, it matters how this sequence of bytes
is stored in the memory.
The two possibilities are <em>little endian</em> and <em>big endian</em>.</p>
<p>But first, let's recap, how hexadecimal values are written and interpreted
and how their bits are stored.
The following variable <code>i</code> is of type <code>integer</code> and uses four bytes of storage:</p>
<div class="highlight"><pre><span class="kt">int</span> <span class="n">i</span> <span class="o">=</span> <span class="mh">0x12345678</span><span class="p">;</span>
</pre></div>
<p>The digits are called <em>nibble</em>, each having a value of 0 to f (representing 15).
In this example, the 8 is the least significant digit, its value is factored with
<math><msup><mn>16</mn><mn>0</mn></msup><mo>=</mo><mn>1</mn></math>.
The next digit from the right, 7, must be multiplied with <math><msup><mn>16</mn><mn>1</mn></msup><mo>=</mo><mn>16</mn></math>,
thus it adds the value <math><mn>7</mn><mo>⋅</mo><mn>16</mn><mo>=</mo><mn>112</mn></math>.
The other nibbles are handled accordingly up to the leftmost (8th) place (value 1), having a factor of
<math><msup><mn>16</mn><mn>7</mn></msup><mo>=</mo><mn>268 435 456</mn></math>.</p>
<p>Each nibble can easily be converted to binary because there are only 16 different values.
For example: 8 = <code>0b1000</code>.
When those are written piece by piece, the hex value can be converted to binary:</p>
<div class="highlight"><pre>0x12345678
= 0x 1 2 3 4 5 6 7 8
= 0b 0001 0010 0011 0100 0101 0110 0111 1000
</pre></div>
<p>As in decimal numbers, the least significant bit is far to the right and the most significant bit is left.
Bitfields are usually displayed with bit 0 (the least significant) to the right and
with increasing bit positions to the left.</p>
<h1 id="little-endian">Little endian</h1>
<p>Systems using little endian byte order store the least significant byte at the lowest address.</p>
<div class="highlight"><pre><span class="kt">int</span> <span class="n">i</span> <span class="o">=</span> <span class="mh">0x12345678</span><span class="p">;</span>
</pre></div>
<p>As in decimal numbers, the rightmost digit (8) is the least significant
and the leftmost digit (1) is the most significant.
Two hexadecimal digits (two nibbles) are stored in one byte:</p>
<table style="border: solid;">
<tr>
<td>↑ large adresses</td><td>0x1003</td>
<td align="center" style="border: solid;">12<span style="vertical-align: sub; font-size: smaller;">hex</span></td>
<td>most significant byte</td>
</tr>
<tr>
<td> </td><td>0x1002</td>
<td align="center" style="border: solid;">34<span style="vertical-align: sub; font-size: smaller;">hex</span></td>
<td> </td>
</tr>
<tr>
<td> </td><td>0x1001</td>
<td align="center" style="border: solid;">56<span style="vertical-align: sub; font-size: smaller;">hex</span></td>
<td> </td>
</tr>
<tr>
<td>↓ small adresses</td><td>0x1000</td>
<td align="center" style="border: solid;">78<span style="vertical-align: sub; font-size: smaller;">hex</span></td>
<td>least significant byte</td>
</tr>
</table>
<p>This becomes twisted, if multiple bytes are displayed in a row.
If the bytes in the row are numbered increasing from right to left, the above sequence of digits
can be recognized easily:</p>
<table>
<thead>
<tr>
<th align="right">0x100f</th>
<th>0e</th>
<th>0d</th>
<th>0c</th>
<th>0b</th>
<th>0a</th>
<th>09</th>
<th>08</th>
<th>07</th>
<th>06</th>
<th>05</th>
<th>04</th>
<th>03</th>
<th>02</th>
<th>01</th>
<th>0x1000</th>
</tr>
</thead>
<tbody>
<tr>
<td align="right">00</td>
<td>00</td>
<td>00</td>
<td>00</td>
<td>00</td>
<td>00</td>
<td>00</td>
<td>00</td>
<td>00</td>
<td>00</td>
<td>00</td>
<td>00</td>
<td>12</td>
<td>34</td>
<td>56</td>
<td>78</td>
</tr>
</tbody>
</table>
<p>But if the bytes are numbered (more intuitively) from left to right,
the sequence of pairs is reversed:</p>
<table>
<thead>
<tr>
<th align="right">0x1000</th>
<th>01</th>
<th>02</th>
<th>03</th>
<th>04</th>
<th>05</th>
<th>06</th>
<th>07</th>
<th>08</th>
<th>09</th>
<th>0a</th>
<th>0b</th>
<th>0c</th>
<th>0d</th>
<th>0e</th>
<th>0x100f</th>
</tr>
</thead>
<tbody>
<tr>
<td align="right">78</td>
<td>56</td>
<td>34</td>
<td>12</td>
<td>00</td>
<td>00</td>
<td>00</td>
<td>00</td>
<td>00</td>
<td>00</td>
<td>00</td>
<td>00</td>
<td>00</td>
<td>00</td>
<td>00</td>
<td>00</td>
</tr>
</tbody>
</table>
<p>Note, that only the bytes (pairs of hexadecimal digits) are in a different sequence,
each pair for itself remains with its less significant digit on the right side
(similar to decimal numbers like 42).
Some Hex-Viewers show pairs of bytes (<code>short</code>, words) -- when these are
stored in little endian format and displayed from left to right, they show up: <code>5678 1234</code>.</p>
<p>The first version appears preferable, at least for multi-byte integers.
With <em>character strings</em>, this is different:</p>
<div class="highlight"><pre><span class="kt">char</span> <span class="n">str</span><span class="p">[]</span> <span class="o">=</span> <span class="s">"Hello world"</span><span class="p">;</span>
</pre></div>
<p>This is comparable to an array with the letter <code>H</code> in the first element,
i.e. the lowest address.
Placing this string behind the variable <code>i</code> and showing addresses
increasing from right to left:</p>
<table>
<thead>
<tr>
<th align="right">0x100f</th>
<th>0e</th>
<th>0d</th>
<th>0c</th>
<th>0b</th>
<th>0a</th>
<th>09</th>
<th>08</th>
<th>07</th>
<th>06</th>
<th>05</th>
<th>04</th>
<th>03</th>
<th>02</th>
<th>01</th>
<th>0x1000</th>
</tr>
</thead>
<tbody>
<tr>
<td align="right">'\0'</td>
<td>'d'</td>
<td>'l'</td>
<td>'r'</td>
<td>'o'</td>
<td>'w'</td>
<td>' '</td>
<td>'o'</td>
<td>'l'</td>
<td>'l'</td>
<td>'e'</td>
<td>'H'</td>
<td>12</td>
<td>34</td>
<td>56</td>
<td>78</td>
</tr>
</tbody>
</table>
<p>In the case of strings, increasing addresses from left to right (as one reads english text)
is favorable (by twisting the integer, again):</p>
<table>
<thead>
<tr>
<th align="right">0x1000</th>
<th>01</th>
<th>02</th>
<th>03</th>
<th>04</th>
<th>05</th>
<th>06</th>
<th>07</th>
<th>08</th>
<th>09</th>
<th>0a</th>
<th>0b</th>
<th>0c</th>
<th>0d</th>
<th>0e</th>
<th>0x100f</th>
</tr>
</thead>
<tbody>
<tr>
<td align="right">78</td>
<td>56</td>
<td>34</td>
<td>12</td>
<td>'H'</td>
<td>'e'</td>
<td>'l'</td>
<td>'l'</td>
<td>'o'</td>
<td>' '</td>
<td>'w'</td>
<td>'o'</td>
<td>'r'</td>
<td>'l'</td>
<td>'d'</td>
<td>'\n'</td>
</tr>
</tbody>
</table>
<h1 id="big-endian">Big endian</h1>
<p>In big endian, the least significant byte is stored at the largest address:</p>
<table style="border: solid;">
<tr>
<td>↑ large adresses</td><td>0x1003</td>
<td align="center" style="border: solid;">78<span style="vertical-align: sub; font-size: smaller;">hex</span></td>
<td>least significant byte</td>
</tr>
<tr>
<td> </td><td>0x1002</td>
<td align="center" style="border: solid;">56<span style="vertical-align: sub; font-size: smaller;">hex</span></td>
<td> </td>
</tr>
<tr>
<td> </td><td>0x1001</td>
<td align="center" style="border: solid;">34<span style="vertical-align: sub; font-size: smaller;">hex</span></td>
<td> </td>
</tr>
<tr>
<td>↓ small adresses</td><td>0x1000</td>
<td align="center" style="border: solid;">12<span style="vertical-align: sub; font-size: smaller;">hex</span></td>
<td>most significant byte</td>
</tr>
</table>
<p>In this byte order, addresses are usually displayed increasing from left
to right as this allows to read the multi-byte integer as well as the string:</p>
<table>
<thead>
<tr>
<th align="right">0x1000</th>
<th>.1</th>
<th>.2</th>
<th>.3</th>
<th>.4</th>
<th>.5</th>
<th>.6</th>
<th>.7</th>
<th>.8</th>
<th>.9</th>
<th>.a</th>
<th>.b</th>
<th>.c</th>
<th>.d</th>
<th>.e</th>
<th>0x100f</th>
</tr>
</thead>
<tbody>
<tr>
<td align="right">12</td>
<td>34</td>
<td>56</td>
<td>78</td>
<td>'H'</td>
<td>'e'</td>
<td>'l'</td>
<td>'l'</td>
<td>'o'</td>
<td>' '</td>
<td>'w'</td>
<td>'o'</td>
<td>'r'</td>
<td>'l'</td>
<td>'d'</td>
<td>'\n'</td>
</tr>
</tbody>
</table>
<h1 id="network-byte-order">Network byte order</h1>
<p>Documents, that are exchanged between systems and especially network transmissions
should care for the byte order.
In the internet protocols, a <em>network byte order</em> is defined (which is big endian).
There are functions to convert network byte order to the host byte order:</p>
<div class="highlight"><pre><span class="cp">#include <netinet/in.h></span>
<span class="kt">unsigned</span> <span class="kt">long</span> <span class="n">htonl</span><span class="p">(</span><span class="kt">unsigned</span> <span class="kt">long</span> <span class="n">hostlong</span><span class="p">)</span> <span class="c1">// host to network, long (32 bit)</span>
<span class="kt">unsigned</span> <span class="kt">long</span> <span class="n">ntohl</span><span class="p">(</span><span class="kt">unsigned</span> <span class="kt">long</span> <span class="n">netlong</span><span class="p">)</span> <span class="c1">// network to host, long (32 bit)</span>
</pre></div>
<h1 id="programming">Programming</h1>
<p>When does a program need to care for endianess?</p>
<p>Of course, when exchanging data with other instances (other programs or the same program running
on a different system) either via files or network,
the byte order matters.
Only if all systems use the same byte order (for example, all are x86 systems),
it can be ignored.</p>
<p>The internet protocol (BSD sockets) libraries use network byte order
and require the IP address to be converted with <code>htonl()</code>.</p>
<p>In internal data structures, the byte order matters if <code>union</code>s or pointers
are used to access portions of other variables.
As long as only math operations and casts are used,
it can be ignored:</p>
<div class="highlight"><pre><span class="k">union</span> <span class="p">{</span>
<span class="kt">uint32_t</span> <span class="n">u32</span><span class="p">;</span>
<span class="kt">uint8_t</span> <span class="n">u8</span><span class="p">[</span><span class="mi">4</span><span class="p">];</span>
<span class="p">}</span> <span class="n">demo</span><span class="p">;</span>
<span class="n">demo</span><span class="p">.</span><span class="n">u32</span> <span class="o">=</span> <span class="mh">0x12345678</span><span class="p">;</span>
<span class="cm">/*</span>
<span class="cm"> * using the address, it depends on the byte ordering, what comes out.</span>
<span class="cm"> */</span>
<span class="n">printf</span><span class="p">(</span><span class="s">"lowest address: u8[0] = %hhx </span><span class="se">\n</span><span class="s">"</span><span class="p">,</span> <span class="n">demo</span><span class="p">.</span><span class="n">u8</span><span class="p">[</span><span class="mi">0</span><span class="p">]);</span>
<span class="n">printf</span><span class="p">(</span><span class="s">"highest address: u8[3] = %hhx </span><span class="se">\n</span><span class="s">"</span><span class="p">,</span> <span class="n">demo</span><span class="p">.</span><span class="n">u8</span><span class="p">[</span><span class="mi">3</span><span class="p">]);</span>
<span class="cm">/*</span>
<span class="cm"> * using math operations, the least significant byte can be masked or calculated</span>
<span class="cm"> * independently from the byte ordering.</span>
<span class="cm"> */</span>
<span class="n">printf</span><span class="p">(</span><span class="s">"least significant byte: %hhx </span><span class="se">\n</span><span class="s">"</span><span class="p">,</span> <span class="n">demo</span><span class="p">.</span><span class="n">u32</span> <span class="o">%</span> <span class="mi">256</span><span class="p">);</span> <span class="c1">// modulo </span>
<span class="n">printf</span><span class="p">(</span><span class="s">"least significant byte: %hhx </span><span class="se">\n</span><span class="s">"</span><span class="p">,</span> <span class="n">demo</span><span class="p">.</span><span class="n">u32</span> <span class="o">&</span> <span class="mh">0xff</span><span class="p">);</span> <span class="c1">// bitwise AND</span>
<span class="n">printf</span><span class="p">(</span><span class="s">"most significant byte: %hhx </span><span class="se">\n</span><span class="s">"</span><span class="p">,</span> <span class="n">demo</span><span class="p">.</span><span class="n">u32</span> <span class="o">/</span> <span class="p">(</span><span class="mi">256</span> <span class="o">*</span> <span class="mi">256</span> <span class="o">*</span> <span class="mi">256</span><span class="p">));</span> <span class="c1">// division</span>
<span class="n">printf</span><span class="p">(</span><span class="s">"most significant byte: %hhx </span><span class="se">\n</span><span class="s">"</span><span class="p">,</span> <span class="n">demo</span><span class="p">.</span><span class="n">u32</span> <span class="o">>></span> <span class="mi">24</span><span class="p">);</span> <span class="c1">// bit shift</span>
</pre></div>
<p>(download <a href="static/byteordering.c">this code</a>.)</p>
<h1 id="further-reading">Further reading</h1>
<ul>
<li><a href="http://fgiesen.wordpress.com/2014/10/25/little-endian-vs-big-endian/">Little-endian vs. big-endian</a>
(discusses trade-offs and current trends)</li>
</ul>Using external files2013-09-12T15:06:00+02:00Georg Wassentag:www.wassen.net,2013-09-12:using-external-files.html<h1 id="motivation">Motivation</h1>
<p>In my thesis, I have a number of citations of a single <a href="http://lwn.net">website</a>
and I don't like to have all of them in the literature (biblatex).
First, I wrote a macro to format those citations uniformly:</p>
<div class="highlight"><pre><span class="c">% use: \lwn{article-no}{author}{title}{date}</span>
<span class="k">\newcommand</span><span class="nb">{</span><span class="k">\lwn</span><span class="nb">}</span>[4]<span class="nb">{</span>[LWN:#1]<span class="k">\footnote</span><span class="nb">{</span>#2: #3 (#4)
<span class="k">\url</span><span class="nb">{</span>http://lwn.net/Articles/#1<span class="nb">}}}</span>
</pre></div>
<p>This allows to write:</p>
<div class="highlight"><pre>In the merge window of Linux 3.12, several new features are
planned <span class="k">\lwn</span><span class="nb">{</span>565251<span class="nb">}{</span>Jonathan Corbet<span class="nb">}{</span>The 3.12 merge window opens<span class="nb">}{</span>September 5, 2013<span class="nb">}</span>...
</pre></div>
<p>This will put a <code>[LWN:565251]</code> in the text and creates a footnote with the name of the author,
the title, date, and URL.</p>
<p>To create a listing of all such citations in the appendix,
the benefit of creating a macro plays out:
by just changing the macro, it is possible to automatically collect
all citations.</p>
<h1 id="index">Index</h1>
<p>With the <code>\index{}</code> macro, it is possible to create entries to the <code>*.idx</code> file.
Those could be postprocessed with a script to generate a piece of LaTeX
that can be included in the document.
But the entries will still occur in the index (or the file must be postprocessed
to remove them). </p>
<p>I used this in a Beamer presentation for a LaTeX tutorial to collect a list of all
packages and all TeXdocs for the handout.
For the Beamer slides, the index was not used otherwise.</p>
<div class="highlight"><pre><span class="k">\makeindex</span>
<span class="k">\newcommand</span><span class="nb">{</span><span class="k">\package</span><span class="nb">}</span>[1]<span class="nb">{</span><span class="k">\texttt</span><span class="nb">{</span>#1<span class="nb">}</span><span class="k">\index</span><span class="nb">{</span>package!#1<span class="nb">}}</span>
<span class="k">\newcommand</span><span class="nb">{</span><span class="k">\texdoc</span><span class="nb">}</span>[1]<span class="nb">{</span><span class="k">\textsc</span><span class="nb">{</span>#1<span class="nb">}</span><span class="k">\index</span><span class="nb">{</span>texdoc!#1<span class="nb">}}</span>
You can use <span class="k">\package</span><span class="nb">{</span>tikz<span class="nb">}</span>, see also <span class="k">\texdoc</span><span class="nb">{</span>latex2e<span class="nb">}</span>.
</pre></div>
<p>In the Makefile was added:</p>
<div class="highlight"><pre>grep <span class="s1">'package!'</span> slides.idx <span class="p">|</span> sed -e <span class="s1">'s/\\indexentry{package!\(.*\)}{.*}/\1/'</span> <span class="p">|</span>sort<span class="p">|</span>uniq > packages.txt
grep <span class="s1">'texdoc!'</span> slides.idx <span class="p">|</span> sed -e <span class="s1">'s/\\indexentry{texdoc!\(.*\)}{.*}/\1/'</span> <span class="p">|</span>sort<span class="p">|</span>uniq > texdoc.txt
</pre></div>
<h1 id="writing-to-files">Writing to files</h1>
<p>Latex allows to open a new file and write (or read) to (from) that file.</p>
<div class="highlight"><pre><span class="k">\newwrite\outputstream</span>
<span class="k">\immediate\openout\outputstream</span>=myfile.tmp
<span class="k">\immediate\write\outputstream</span><span class="nb">{</span>foo 1<span class="nb">}</span>
<span class="k">\immediate\write\outputstream</span><span class="nb">{</span>foo 2<span class="nb">}</span>
<span class="k">\immediate\write\outputstream</span><span class="nb">{</span><span class="k">\string\textbf</span><span class="nb">{</span>foo 3<span class="nb">}}</span>
<span class="k">\immediate\closeout\outputstream</span>
</pre></div>
<p>(Source of that example: <a href="http://www.latex-community.org/viewtopic.php?f=5&t=1046">Juanjo (latex-community)</a>).</p>
<p>My new macro for the <code>\lwn</code> citation is now:</p>
<div class="highlight"><pre><span class="c">% use: \lwn{article-no}{author}{title}{date}</span>
<span class="k">\newcommand</span><span class="nb">{</span><span class="k">\lwn</span><span class="nb">}</span>[4]<span class="nb">{</span>[LWN:#1]<span class="k">\footnote</span><span class="nb">{</span>#2: #3 (#4) <span class="k">\url</span><span class="nb">{</span>http://lwn.net/Articles/#1<span class="nb">}}</span><span class="k">\immediate\write\outputstream</span><span class="nb">{</span>#1;#2;#3;#4<span class="nb">}}</span>
<span class="c">% Open file at the begin of the document</span>
<span class="k">\AtBeginDocument</span><span class="nb">{</span><span class="c">%</span>
<span class="k">\newwrite\outputstream</span>
<span class="k">\immediate\openout\outputstream</span>=lwncite.csv
<span class="nb">}</span>
<span class="c">% Close the file at the end of document</span>
<span class="k">\AtEndDocument</span><span class="nb">{</span><span class="c">%</span>
<span class="k">\immediate\closeout\outputstream</span>
<span class="nb">}</span>
</pre></div>
<p>The macros <code>\AtBeginDocument</code> and <code>\AtEndDocument</code> register those pieces (opening and closing the file)
for the beginning and end of the document.</p>
<p>The file is <code>lwncite.csv</code> containing lines with article number, author, title, and date,
separated by semicolon.
In the Makefile, this is sorted and reformated:</p>
<div class="highlight"><pre>sort lwncite.csv <span class="p">|</span> awk -F<span class="s1">';'</span> <span class="s1">'{print "[LWN:" $1 "] -- " $2 ". ``" $3 "''. http://lwn.net/Articles/" $1 " (" $4 ")"}'</span> > lwncite.txt
</pre></div>
<p>This will be changed from text format to LaTeX as soon as I integrate the file into my document.</p>
<h1 id="fix-no-room-for-a-new-write">Fix "No room for a new <code>\write</code>"</h1>
<p>If the compilation breaks with the error message "No room for a new <code>\write</code>",
the maximum number of 16 open files is exceeded. But help is simple:
The Package <code>morewrites</code> can be loaded at the beginning of the document
(before any other packages). It invades deeply into the guts of LaTeX,
but it has helped me without further problems. <a href="http://www.ctan.org/pkg/morewrites">Source</a></p>Using SQLite in C programs2013-09-02T11:18:00+02:00Georg Wassentag:www.wassen.net,2013-09-02:sqlite-c.html<h1 id="intro">Intro</h1>
<p><a href="http://www.sqlite.org">SQLite</a> is a free local database with SQL interface.
I've written an <a href="http://www.wassen.net/sqlite.html">introduction</a> for the basics.</p>
<p>For interfacing with C, SQLite offers two files: a C file (module) containing all functions
and a header file.
They can either be included with the own files and compiled in
or used as a shared library.
Distributions should ship a <code>-devel</code> package for this.
The version 2 is deprecated and should be replaced with version 3.</p>
<p>The most important data structures are <code>sqlite3</code> for the database connection
(similar to a file handle, it is opened, used for reading and writing and finally closed)
and <code>sqlite3_stmt</code> for the queries.
Queries are prepared, executed ("step"ed) and finalized.</p>
<h1 id="the-database-connection">The Database connection</h1>
<p>A database is stored in a single binary file.
Temporary data can be held solely in memory by using the special file name <code>:memory:</code>.
The latter allows using SQLite for managing data, e.g. sorting.</p>
<p>The following program opens a database and closes it again.
The return value is checked and an error message is printed if something went wrong.</p>
<div class="highlight"><pre><span class="cp">#include <stdio.h> </span><span class="c1">// printf</span>
<span class="cp">#include <sqlite3.h> </span><span class="c1">// SQLite header (from /usr/include)</span>
<span class="kt">int</span> <span class="nf">main</span><span class="p">()</span>
<span class="p">{</span>
<span class="n">sqlite3</span> <span class="o">*</span><span class="n">db</span><span class="p">;</span> <span class="c1">// database connection</span>
<span class="kt">int</span> <span class="n">rc</span><span class="p">;</span> <span class="c1">// return code</span>
<span class="kt">char</span> <span class="o">*</span><span class="n">errmsg</span><span class="p">;</span> <span class="c1">// pointer to an error string</span>
<span class="cm">/*</span>
<span class="cm"> * open SQLite database file test.db</span>
<span class="cm"> * use ":memory:" to use an in-memory database</span>
<span class="cm"> */</span>
<span class="n">rc</span> <span class="o">=</span> <span class="n">sqlite3_open</span><span class="p">(</span><span class="s">":memory:"</span><span class="p">,</span> <span class="o">&</span><span class="n">db</span><span class="p">);</span>
<span class="k">if</span> <span class="p">(</span><span class="n">rc</span> <span class="o">!=</span> <span class="n">SQLITE_OK</span><span class="p">)</span> <span class="p">{</span>
<span class="n">printf</span><span class="p">(</span><span class="s">"ERROR opening SQLite DB in memory: %s</span><span class="se">\n</span><span class="s">"</span><span class="p">,</span> <span class="n">sqlite3_errmsg</span><span class="p">(</span><span class="n">db</span><span class="p">));</span>
<span class="k">goto</span> <span class="n">out</span><span class="p">;</span>
<span class="p">}</span>
<span class="n">printf</span><span class="p">(</span><span class="s">"opened SQLite handle successfully.</span><span class="se">\n</span><span class="s">"</span><span class="p">);</span>
<span class="cm">/* use the database... */</span>
<span class="nl">out</span><span class="p">:</span>
<span class="cm">/*</span>
<span class="cm"> * close SQLite database</span>
<span class="cm"> */</span>
<span class="n">sqlite3_close</span><span class="p">(</span><span class="n">db</span><span class="p">);</span>
<span class="n">printf</span><span class="p">(</span><span class="s">"database closed.</span><span class="se">\n</span><span class="s">"</span><span class="p">);</span>
<span class="p">}</span>
</pre></div>
<p>This can be compiled using (provided, that <code>sqlite3-devel</code> is installed):</p>
<div class="highlight"><pre><span class="nv">$gcc</span> -O0 -g openclose.c -lsqlite3 -o openclose
</pre></div>
<p>A Makefile for this could be (this assumes, that the above file is saved as <code>openclose.c</code>,
that <code>pkg-config</code> and <code>sqlite3-devel</code> are installed and uses the build-in rules to
create executables from C files):</p>
<div class="highlight"><pre><span class="nv">CFLAGS</span><span class="o">=</span><span class="k">$(</span>shell pkg-config --cflags sqlite3<span class="k">)</span> -O0 -g
<span class="nv">LDLIBS</span><span class="o">=</span><span class="k">$(</span>shell pkg-config --libs sqlite3<span class="k">)</span>
<span class="nf">default </span><span class="o">:</span> <span class="n">openclose</span>
<span class="nf">debug </span><span class="o">:</span>
@echo <span class="nv">CFLAGS</span> <span class="o">=</span> <span class="k">$(</span>CFLAGS<span class="k">)</span>
@echo <span class="nv">LDLIBS</span> <span class="o">=</span> <span class="k">$(</span>LDLIBS<span class="k">)</span>
</pre></div>
<h1 id="the-classic-way">The classic way</h1>
<p>The basic working is always:</p>
<ul>
<li>Prepare a query. This is usually a string (e.g. created with <code>asprintf()</code>)
that can optionally contain placeholders for binding values.
This generates a <em>statement</em>.</li>
<li>Execute the statement, either once (for creating tables, inserting, or updating)
or until all resulting rows are read.</li>
<li>Free the ressources by <em>finalizing</em> the statement.</li>
</ul>
<h2 id="executing-a-static-query">Executing a static query</h2>
<p>The following example creates a table.
The query is a static string.
This piece of code can be placed between <code>sqlite3_open()</code> and <code>sqlite3_close()</code> in the above example.</p>
<div class="highlight"><pre><span class="n">sqlite3_stmt</span> <span class="o">*</span><span class="n">stmt</span><span class="p">;</span> <span class="cm">/* 1 */</span>
<span class="n">sqlite3_prepare_v2</span><span class="p">(</span><span class="n">db</span><span class="p">,</span> <span class="s">"CREATE TABLE demo (name TEXT, age INTEGER);"</span><span class="p">,</span> <span class="o">-</span><span class="mi">1</span><span class="p">,</span> <span class="o">&</span><span class="n">stmt</span><span class="p">,</span> <span class="nb">NULL</span><span class="p">);</span> <span class="cm">/* 2 */</span>
<span class="n">rc</span> <span class="o">=</span> <span class="n">sqlite3_step</span><span class="p">(</span><span class="n">stmt</span><span class="p">);</span> <span class="cm">/* 3 */</span>
<span class="k">if</span> <span class="p">(</span><span class="n">rc</span> <span class="o">!=</span> <span class="n">SQLITE_DONE</span><span class="p">)</span> <span class="p">{</span>
<span class="n">printf</span><span class="p">(</span><span class="s">"ERROR inserting data: %s</span><span class="se">\n</span><span class="s">"</span><span class="p">,</span> <span class="n">sqlite3_errmsg</span><span class="p">(</span><span class="n">db</span><span class="p">));</span>
<span class="k">goto</span> <span class="n">out</span><span class="p">;</span>
<span class="p">}</span>
<span class="n">sqlite3_finalize</span><span class="p">(</span><span class="n">stmt</span><span class="p">);</span> <span class="cm">/* 4 */</span>
</pre></div>
<ol>
<li>A variable for the statement is allocated.</li>
<li>The current version of the <em>prepare</em> function is <code>sqlite3_prepare_v2()</code>.<ul>
<li>The first parameter is the database connection <code>db</code>.</li>
<li>The second parameter is the query (UTF-8 string).</li>
<li>The third parameter should be the length of the query but can be <code>-1</code> for zero-terminated C strings.</li>
<li>For <code>stmt</code>, the address must be given because a pointer is returned therein.</li>
<li>The final parameter is not used here, please consult the <a href="http://www.sqlite.org/c3ref/prepare.html">documentation</a>.</li>
</ul>
</li>
<li>The function <code>sqlite3_step</code> executes the prepared <em>statement</em>.
Here, no returning rows are expected, only the success is tested.</li>
<li>Free the ressources of the prepared statement.</li>
</ol>
<h2 id="execute-a-dynamic-query">Execute a dynamic query</h2>
<p>The next example uses <code>asprintf()</code> to dynamically create a query.
This can be done with simple and trusted input.
For <strong>content provided by users</strong>, the binding of values should be used as shown below.</p>
<div class="highlight"><pre><span class="kt">char</span> <span class="o">*</span><span class="n">query</span> <span class="o">=</span> <span class="nb">NULL</span><span class="p">;</span>
<span class="n">asprintf</span><span class="p">(</span><span class="o">&</span><span class="n">query</span><span class="p">,</span> <span class="s">"insert into demo (name, age) values ('%s', %d);"</span><span class="p">,</span> <span class="s">"Tom"</span><span class="p">,</span> <span class="mi">20</span><span class="p">);</span> <span class="cm">/* 1 */</span>
<span class="n">sqlite3_prepare_v2</span><span class="p">(</span><span class="n">db</span><span class="p">,</span> <span class="n">query</span><span class="p">,</span> <span class="n">strlen</span><span class="p">(</span><span class="n">query</span><span class="p">),</span> <span class="o">&</span><span class="n">stmt</span><span class="p">,</span> <span class="nb">NULL</span><span class="p">);</span> <span class="cm">/* 2 */</span>
<span class="n">rc</span> <span class="o">=</span> <span class="n">sqlite3_step</span><span class="p">(</span><span class="n">stmt</span><span class="p">);</span>
<span class="k">if</span> <span class="p">(</span><span class="n">rc</span> <span class="o">!=</span> <span class="n">SQLITE_DONE</span><span class="p">)</span> <span class="p">{</span>
<span class="n">printf</span><span class="p">(</span><span class="s">"ERROR inserting data: %s</span><span class="se">\n</span><span class="s">"</span><span class="p">,</span> <span class="n">sqlite3_errmsg</span><span class="p">(</span><span class="n">db</span><span class="p">));</span>
<span class="k">goto</span> <span class="n">out</span><span class="p">;</span>
<span class="p">}</span>
<span class="n">sqlite3_finalize</span><span class="p">(</span><span class="n">stmt</span><span class="p">);</span>
<span class="n">free</span><span class="p">(</span><span class="n">query</span><span class="p">);</span> <span class="cm">/* 3 */</span>
</pre></div>
<ol>
<li>The function <code>asprintf()</code> allocates a buffer large enough to hold the created string.</li>
<li>The statement is now prepared from the <code>query</code> string. The length can be given (it should be faster).</li>
<li>Don't forget to free the string allocated by <code>asprintf()</code>.</li>
</ol>
<p>Once again: if a user provides vile input (i.e. <a href="http://en.wikipedia.org/wiki/SQL_injection">"SQL injection"</a>), the database is in danger.</p>
<h2 id="binding-values">Binding values</h2>
<p>The following example shows how to use place-holders and bind values to them.</p>
<div class="highlight"><pre><span class="n">sqlite3_prepare_v2</span><span class="p">(</span><span class="n">db</span><span class="p">,</span> <span class="s">"insert into demo (name, age) values (?1, ?2);"</span><span class="p">,</span> <span class="o">-</span><span class="mi">1</span><span class="p">,</span> <span class="o">&</span><span class="n">stmt</span><span class="p">,</span> <span class="nb">NULL</span><span class="p">);</span> <span class="cm">/* 1 */</span>
<span class="n">sqlite3_bind_text</span><span class="p">(</span><span class="n">stmt</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="s">"Susan"</span><span class="p">,</span> <span class="o">-</span><span class="mi">1</span><span class="p">,</span> <span class="n">SQLITE_STATIC</span><span class="p">);</span> <span class="cm">/* 2 */</span>
<span class="n">sqlite3_bind_int</span><span class="p">(</span><span class="n">stmt</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">21</span><span class="p">);</span> <span class="cm">/* 3 */</span>
<span class="n">rc</span> <span class="o">=</span> <span class="n">sqlite3_step</span><span class="p">(</span><span class="n">stmt</span><span class="p">);</span>
<span class="k">if</span> <span class="p">(</span><span class="n">rc</span> <span class="o">!=</span> <span class="n">SQLITE_DONE</span><span class="p">)</span> <span class="p">{</span>
<span class="n">printf</span><span class="p">(</span><span class="s">"ERROR inserting data: %s</span><span class="se">\n</span><span class="s">"</span><span class="p">,</span> <span class="n">sqlite3_errmsg</span><span class="p">(</span><span class="n">db</span><span class="p">));</span>
<span class="k">goto</span> <span class="n">out</span><span class="p">;</span>
<span class="p">}</span>
<span class="n">sqlite3_finalize</span><span class="p">(</span><span class="n">stmt</span><span class="p">);</span>
</pre></div>
<ol>
<li>Instead of strings and integers, use <code>?1</code> etc. as place-holders.
Others are possible, see the <a href="http://www.sqlite.org/c3ref/bind_blob.html">documentation</a>.</li>
<li>Bind the string "Susan" to <code>?1</code>. As this is a static string, tell SQLite not to free it.
If it is an allocated string, a function can be given to free it after the statement is processed.</li>
<li>Bind an integer value (no freeing required).</li>
</ol>
<p>The rest is as before.</p>
<h2 id="query-data-with-the-classic-method">Query data with the classic method</h2>
<p>If the prepared statement returns resulting rows,
the function <code>sqlite3_step()</code> returns the value <code>SQLITE_ROW</code> and can be called until no more data is available.
The columns can be accessed by index:</p>
<div class="highlight"><pre>sqlite3_prepare_v2(db, "select distinct name, age from demo where age > ? order by 2,1;", -1,
&stmt, NULL);
sqlite3_bind_int(stmt, 1, 16); /* 1 */
while ( (rc = sqlite3_step(stmt)) == SQLITE_ROW) { /* 2 */
printf("%s is %d years old\n", sqlite3_column_text(stmt, 0), sqlite3_column_int(stmt, 1)); /* 3 */
}
sqlite3_finalize(stmt);
</pre></div>
<ol>
<li>The prepared query contains <code>?1</code> in the <em>WHERE</em> clause. Bind an integer value.</li>
<li>Loop while more data is available.</li>
<li>The data is accessed by column index. Strings are given as <code>const char *</code> and can be directly given to
functions like <code>printf()</code>.</li>
</ol>
<h1 id="one-step-query-execution-interface">One-Step Query Execution Interface</h1>
<p>The convenience function <a href="http://www.sqlite.org/c3ref/exec.html"><code>sqlite3_exec()</code></a> combines the above steps.
However, the binding of values appears not to be implemented.
For user input, use either very careful masking or the classic way.
The following example shows how to execute two SQL queries in one call.
Resulting data is given to a callback function.</p>
<div class="highlight"><pre><span class="kt">char</span> <span class="o">*</span><span class="n">errmsg</span><span class="p">;</span>
<span class="kt">int</span> <span class="nf">callback</span><span class="p">(</span><span class="kt">void</span> <span class="o">*</span><span class="n">arg</span><span class="p">,</span> <span class="kt">int</span> <span class="n">argc</span><span class="p">,</span> <span class="kt">char</span> <span class="o">**</span><span class="n">argv</span><span class="p">,</span> <span class="kt">char</span> <span class="o">**</span><span class="n">colName</span><span class="p">)</span> <span class="p">{</span> <span class="cm">/* 1 */</span>
<span class="kt">int</span> <span class="n">i</span><span class="p">;</span>
<span class="k">for</span><span class="p">(</span><span class="n">i</span><span class="o">=</span><span class="mi">0</span><span class="p">;</span> <span class="n">i</span><span class="o"><</span><span class="n">argc</span><span class="p">;</span> <span class="n">i</span><span class="o">++</span><span class="p">){</span>
<span class="n">printf</span><span class="p">(</span><span class="s">"%s = %s</span><span class="se">\t</span><span class="s">"</span><span class="p">,</span> <span class="n">colName</span><span class="p">[</span><span class="n">i</span><span class="p">],</span> <span class="n">argv</span><span class="p">[</span><span class="n">i</span><span class="p">]</span> <span class="o">?</span> <span class="o">:</span> <span class="s">"NULL"</span><span class="p">);</span>
<span class="p">}</span>
<span class="n">printf</span><span class="p">(</span><span class="s">"</span><span class="se">\n</span><span class="s">"</span><span class="p">);</span>
<span class="k">return</span> <span class="mi">0</span><span class="p">;</span>
<span class="p">}</span>
<span class="n">rc</span> <span class="o">=</span> <span class="n">sqlite3_exec</span><span class="p">(</span><span class="n">db</span><span class="p">,</span> <span class="cm">/* 2 */</span>
<span class="s">"select count(*), avg(age) from demo; select distinct name, age from demo order by 1,2;"</span><span class="p">,</span>
<span class="n">callback</span><span class="p">,</span> <span class="nb">NULL</span><span class="p">,</span> <span class="o">&</span><span class="n">errmsg</span><span class="p">);</span>
<span class="k">if</span> <span class="p">(</span><span class="n">errmsg</span> <span class="o">!=</span> <span class="nb">NULL</span><span class="p">)</span> <span class="p">{</span> <span class="cm">/* 3 */</span>
<span class="n">printf</span><span class="p">(</span><span class="s">"Error in sqlite3_exec: %s</span><span class="se">\n</span><span class="s">"</span><span class="p">,</span> <span class="n">errmsg</span><span class="p">);</span>
<span class="n">sqlite3_free</span><span class="p">(</span><span class="n">errmsg</span><span class="p">);</span>
<span class="p">}</span>
</pre></div>
<ol>
<li>Provide a callback function.
It can be given arguments to distinguish multiple uses.
The query results are provided similar to <code>main()</code> with <code>argc</code> and <code>argv</code> plus the names of the columns in a fourth string array.
Here, the example function just prints key=value.</li>
<li>Execute one or multiple queries. The callback function and its arguments (here: <code>NULL</code>)
and a char pointer for an error message are the parameters.</li>
<li>If an error occured, errmsg is allocated and non-<code>NULL</code>.
In this case, it should be freed using the function <code>sqlite2_free()</code>.</li>
</ol>
<h1 id="more-examples">More examples</h1>
<p>More examples are in <a href="download/sqlite.tar.gz">this source code archive</a>.</p>SQLite2013-09-01T19:14:00+02:00Georg Wassentag:www.wassen.net,2013-09-01:sqlite.html<p><a href="http://www.sqlite.org">SQLite</a> is a free database engine that can be build into programs.
There is also a command line tool.
The data is stored in local files.
It is not intended for multi-user or as web-server back-end (but probably can manage these, too),
but for easily interfacing from programs.
It can be used from Shell scripts, C and C++ programs, Python programs and many more.
Many Web-Toolkits (like Django) use it for quickly starting to develop without having to install a database server.</p>
<h1 id="sql">SQL</h1>
<p>The SQL dialect is a bit limited, but for those knowing PostgreSQL, MySQL or Oracle, it is very easy to adapt.
The most important difference is SQLite ignoring data types:
it does not matter what type a column is, it holds all sorts of data.</p>
<div class="highlight"><pre><span class="k">CREATE</span> <span class="k">TABLE</span> <span class="n">demo</span> <span class="p">(</span><span class="n">ID</span> <span class="nb">INTEGER</span> <span class="k">PRIMARY</span> <span class="k">KEY</span><span class="p">,</span> <span class="n">name</span> <span class="nb">TEXT</span><span class="p">,</span> <span class="n">age</span> <span class="nb">INTEGER</span><span class="p">);</span>
<span class="k">INSERT</span> <span class="k">INTO</span> <span class="k">TABLE</span> <span class="n">demo</span> <span class="p">(</span><span class="n">name</span><span class="p">,</span> <span class="n">age</span><span class="p">)</span> <span class="k">VALUES</span> <span class="p">(</span><span class="s1">'Peter'</span><span class="p">,</span> <span class="mi">42</span><span class="p">);</span>
<span class="k">UPDATE</span> <span class="n">demo</span> <span class="k">SET</span> <span class="p">(</span><span class="n">age</span><span class="o">=</span><span class="mi">41</span><span class="p">)</span> <span class="k">WHERE</span> <span class="n">name</span> <span class="o">=</span> <span class="s1">'Peter'</span><span class="p">;</span>
<span class="k">INSERT</span> <span class="k">INTO</span> <span class="n">demo</span> <span class="p">(</span><span class="n">name</span><span class="p">,</span> <span class="n">age</span><span class="p">)</span> <span class="k">VALUES</span> <span class="p">(</span><span class="s1">'Tom'</span><span class="p">,</span> <span class="mi">20</span><span class="p">);</span>
<span class="k">DELETE</span> <span class="k">FROM</span> <span class="n">demo</span> <span class="k">WHERE</span> <span class="n">age</span> <span class="o">></span> <span class="mi">100</span><span class="p">;</span>
<span class="k">SELECT</span> <span class="o">*</span> <span class="k">from</span> <span class="n">demo</span><span class="p">;</span>
<span class="k">SELECT</span> <span class="n">name</span><span class="p">,</span> <span class="n">age</span> <span class="k">FROM</span> <span class="n">demo</span> <span class="k">WHERE</span> <span class="n">age</span> <span class="o">></span> <span class="mi">16</span> <span class="k">ORDER</span> <span class="k">BY</span> <span class="n">age</span><span class="p">;</span>
</pre></div>
<p>If the first column is of the exact data type <code>INTEGER PRIMARY KEY</code>, it will be an alias for <code>rowid</code>.
This is the only column, that holds only integer and automatically increments to a new value for every new row.
Commands must be terminated by a semicolon, the case does not matter.
Strings are given in single quotes.</p>
<p>The SQL dialect is documented on the <a href="http://www.sqlite.org/lang.html">web site</a>.</p>
<h1 id="command-line-tool">Command line tool</h1>
<p>The command line interface <code>sqlite3</code> can be used to interactively control the database file
or to check what your program has done.
The file must be given on the command line.</p>
<div class="highlight"><pre><span class="nv">$ </span>sqlite3 test.db
sqlite> .schema <span class="c"># shows the existing tables and their structure</span>
sqlite> .mode column <span class="c"># switch to a nicer display mode, csv is also possible</span>
sqlite> .header on <span class="c"># show column headers</span>
sqlite> <span class="k">select</span> * from demo<span class="p">;</span>
sqlite> .quit
</pre></div>
<p>Use <code>.help</code> to show what interal commands are understood.</p>
<p>Global settings can be given in a file <code>~/.sqliterc</code>, e.g.</p>
<div class="highlight"><pre>.mode column
.header on
</pre></div>
<h1 id="interfacing-from-shell-scripts">Interfacing from Shell scripts</h1>
<p>A query can be given on the command line:</p>
<div class="highlight"><pre><span class="nv">$ </span>sqlite3 test.db <span class="s2">"select * from demo;"</span><span class="p">;</span>
</pre></div>
<p>The format can be configured to columns (for viewing), csv (for importing data to spreadsheets), HTML and some more.</p>
<p>To avoid stumbling over global settings for human readability,
scripts should always set the output format according.</p>
<h1 id="further-reading">Further reading</h1>
<p>How SQLite itself sees its strengths and weaknesses: <a href="http://www.sqlite.org/whentouse.html">Appropriate Uses For SQLite</a></p>X11 Clipboard2013-08-23T20:07:00+02:00Georg Wassentag:www.wassen.net,2013-08-23:x11-clipboard.html<p>There are two programs helping to handle both methods.
They appear not to be installed by default (at least on OpenSUSE and Fedora).
But the package managers should provide them both.
One is <a href="http://www.vergenet.net/~conrad/software/xsel/">xsel</a>
the other is <a href="http://sourceforge.net/projects/xclip/">xclip</a>.</p>
<h1 id="technical-background">Technical background</h1>
<p>Note, that there are two distinct mechanisms to copy and paste in X11:
the <em>selection</em> and the <em>clipbard</em>.
The <em>selection</em> is used by highlighting some text with the mouse
and pasted with the middle button (which is, on most mice, the wheel).
To be precise, there are <em>primary</em> and <em>secondary</em> selections,
but the latter is hardly used.
The <em>clipboard</em> uses Ctrl-C and Ctrl-V to copy and paste (or the menu entries below Edit).</p>
<p>If I understood correctly, the selection is from the X server.
It is not stored in the server, but only mediated between source and target process.
If the source process ends, the <em>selection</em> can no longer be pasted.
Desktop environments have a clipboard daemon (xclipboard, Klipper on KDE)
that handles the <em>clipboard</em>.</p>
<p>Both programs <code>xsel</code> and <code>xclip</code> keep running in the background to provide their selection content
and exit when another application takes over.</p>
<h1 id="xsel">xsel</h1>
<p>Xsel is easier to use, because it detects if you're using it as input
or ouput:</p>
<div class="highlight"><pre>xsel < file <span class="c"># reads the content of file into the primary selection</span>
xsel > file <span class="c"># writes the primary selection to the file</span>
</pre></div>
<p>By default, the program uses the <em>primary selection</em> (that one with the middle mouse button).
With the parameter <code>-b</code> (or <code>--clipboard</code>), the <em>clipboard</em> (Ctrl-C, Ctrl-V) is used.
The <em>secondary selection</em> can be addressed with <code>-s</code> (<code>--secondary</code>).
The content of <em>primary</em> and <em>secondary</em> selection can be exchanged with <code>xsel -x</code>.
It's also possible to delete the current content of the <em>selection</em> with <code>-c</code>
(only if the source is xsel itself) and <code>-d</code> (requests the source program to discard the selection).
Input can also be appended to the current content with <code>-a</code>.</p>
<p>To keep the content after the program ends, use <code>-k</code>.
According the the manual page,
this requests the data from the current source process and stores it for pasting
while running in the background.</p>
<p>There are options for the X11 interaction (display, etc.),
please refer to the man page if you need them.
For me, the programs works without.</p>
<h1 id="xclip">xclip</h1>
<p>The other program, <code>xclip</code>, is less convenient in day-to-day use.
To get the content of <em>selection</em> or <em>clipbaord</em>, it requires the command line option <code>-o</code>.
Further, the non-default <em>secondary selection</em> and <em>clipboard</em> are addressed with the
long options <code>-selection secondary</code> and <code>-selection clipboard</code>
(both can be abbreviated <code>-se s</code> and <code>-se c</code>).</p>
<p>With <code>-l</code> (<code>--loops</code>),
it can be instructed to wait in the background only for a number of requests from other applications
before terminating. </p>
<h1 id="more-examples">More Examples</h1>
<p>Both should work as expected in shell pipes:
they place their input in the selection/clipboard and print its content when called.</p>
<h2 id="examples-for-the-selection">Examples for the <em>selection</em></h2>
<p>Read a file into the <em>selection</em>:</p>
<div class="highlight"><pre>xsel file
xclip file
</pre></div>
<p>or</p>
<div class="highlight"><pre>xsel < file
xclip < file
</pre></div>
<p>Put something in the <em>selection</em> using a pipe:</p>
<div class="highlight"><pre>some command | xsel
some command | xclip
</pre></div>
<p>Append the output to the <em>selection</em> (xclip can't do that)</p>
<div class="highlight"><pre>some command | xsel -a
</pre></div>
<p>Retreive the <em>selection</em> to the standard output</p>
<div class="highlight"><pre>xsel
xclip -o
</pre></div>
<p>Redirect the <em>selection</em> to a file:</p>
<div class="highlight"><pre>xsel > file
xclip -o > file
</pre></div>
<p>Append the <em>selection</em> to the file:</p>
<div class="highlight"><pre>xsel >> file
xclip -o >> file
</pre></div>
<h2 id="examples-for-the-clipboard">Examples for the <em>clipboard</em></h2>
<p>Put something in the <em>clipboard</em> (e.g. to paste it in a GUI via Ctrl-V):</p>
<div class="highlight"><pre>some command | xsel -b
some command | xclip -selection clipboard
</pre></div>
<p>Retreive something from the <em>clipboard</em> (that was copied by Ctrl-C or Ctrl-X):</p>
<div class="highlight"><pre>xsel -b
xclip -o -selection clipbaord
</pre></div>
<h2 id="network">Network</h2>
<p>It is even possible, to use them over X11-forwarding SSH sessions:
A file read into <code>xsel</code> (or <code>xclip</code>) on a remote system can be retrieved locally and vice-versa:</p>
<div class="highlight"><pre>erde<span class="nv">$ </span>ssh -X sonne
sonne<span class="nv">$ </span>xsel < .bashrc
sonne<span class="nv">$ </span><span class="nb">exit</span>
erde<span class="nv">$ </span>xsel > .bashrc
</pre></div>
<h2 id="note-for-gvim">Note for gvim</h2>
<p>The selection is in the register <code>"*</code>, the clipboard in <code>"+</code>.</p>
<p>Sources: <a href="http://www.commandlinefu.com/commands/view/2586/pipe-output-of-a-command-to-your-clipboard">commandlinefu.com</a>,
<a href="http://www.vergenet.net/~conrad/software/xsel/">xsel project page</a>,
<a href="http://sourceforge.net/projects/xclip/">xclip project page</a> and the man pages.</p>Markdown2013-08-23T09:06:00+02:00Georg Wassentag:www.wassen.net,2013-08-23:markdown.html<p>My Vim installation recognizes the extension <code>.md</code> as Modula.
The filetype (for syntax highlighting) can be changed via</p>
<div class="highlight"><pre>:set filetype=markdown
</pre></div>
<p>In other files, a comment can be used to change this on a per-file basis.
But Markdown does not support comments (or deactivated lines), so this setting
would show up in the final document.
But the following setting in <code>.vimrc</code> does the job:</p>
<div class="highlight"><pre>au BufNewFile,BufRead *.md setlocal ft=markdown
</pre></div>
<p>Other useful settings:</p>
<div class="highlight"><pre>imap <F5> <ESC>yypVr=o
imap <F6> <ESC>yypVr-o
</pre></div>
<p>Those two mappings set F5 and F6 to place equal-signs and dashes below
the current line.
It does so by copying <code>yy</code> and pasting <code>p</code> the current line,
then marking the whole line <code>V</code> and replacing each character <code>r=</code>.
To use this from input mode, it first issues ESC and finally <code>o</code>
to continue writing in the line below.
This helps a lot with the underlined type of headings.</p>SSH2013-08-23T07:52:00+02:00Georg Wassentag:www.wassen.net,2013-08-23:ssh.html<h1 id="basics">Basics</h1>
<p>To log into a system with your current username, just call <code>ssh hostname</code>:</p>
<div class="highlight"><pre>georg@erde:~<span class="nv">$ </span>ssh sonne
Passwort: ***********
georg@sonne~<span class="err">$</span>
</pre></div>
<p><em>Note</em>: the password will not be displayed (not even stars).
If you have a different account on the other system, prepend your
user name like in an e-mail address: <code>ssh gw@sonne</code>.</p>
<p>With the command line option <code>-X</code> (upper case X),
the X11 session will be forwarded over the SSH connection.
When starting a program with GUI,
it will be displayed on your local screen and your input
will be redirected to the programm running on the remote machine.</p>
<h1 id="using-password-less-login">Using password-less login</h1>
<p>SSH can create a pair of public and private keys to log
into systems without everytime entering the password.
Additionally, this comes handy for scripts where you
would never put your password in.</p>
<p>First, create a key pair:</p>
<div class="highlight"><pre><span class="nv">$ </span>ssh-keygen
</pre></div>
<p>It will ask for a name where to store the key files
and for a passphrase.
The <em>phrase</em> should be taken literally: this will be your
master key, don't just provide 8 characters (on none).
I will show how to use the <code>ssh-agent</code>, so this passphrase
will only be entered once after login.
See <code>man ssh-keygen</code> for more options.</p>
<h2 id="distributing-the-public-key">Distributing the public key</h2>
<p>The private key must be kept secret.
By default, it is stored in the hidden directory <code>~/.ssh/</code>
in the file <code>id_rsa</code>
and only readable for the user.
(SSH will reject using it if the file is readable for more than the owner.)</p>
<p>The public key has the extension .pub (<code>id_rsa.pub</code>).
To be accepted, it must be <em>appended</em> to the file <code>~/.ssh/authorized_keys</code>.
If your home directories are synchronized (e.g. via NFS),
just append it to the local file:</p>
<div class="highlight"><pre>~<span class="nv">$ </span><span class="nb">cd</span> .ssh
.ssh<span class="nv">$ </span>cat id_rsa.pub >> authorized_keys
</pre></div>
<p>Otherwise, the public key must be copied to the remote machine.
The tool <code>ssh-copy-id</code> helps: just call it with the hostname
(or name@hostname, if the account is named differently):</p>
<div class="highlight"><pre>georg@erde~<span class="nv">$ </span>ssh-copy-id mond
Password: ***********
Now try loggin into the machine...
georg@erde~<span class="nv">$ </span>ssh mond
georg@mond~<span class="err">$</span>
</pre></div>
<h2 id="using-the-ssh-agent-to-avoid-the-passphrase">Using the SSH-Agent to avoid the passphrase</h2>
<p>If you're following the steps above, so far, you have only
replaced the (probably 8 character) password with a (according
to my suggestion much longer) passphrase.
But as promised, there's help:</p>
<div class="highlight"><pre><span class="nv">$ </span>ssh-agent
<span class="nv">$ </span>ssh-add
Passphrase: *************************
<span class="nv">$ </span>ssh sonne
</pre></div>
<p>The first line starts the SSH-Agent in the background.
Then <code>ssh-add</code> registers your private keys with the agent.
It will ask for the passphrase.
From now on, every subsequent SSH will use the password-less
(and passphrase-less) login.
If your public key is not on the target system,
SSH falls back to asking for the password.</p>
<p>With the command line option <code>-A</code>,
you can even forward the agent to a remote machine
and then login from there to the next machine
without entering a password (of -phrase) again.</p>
<h1 id="more-configuration-for-more-convenience">More configuration for more convenience</h1>
<p>SSH can be configured with the file <code>~/.ssh/config</code>.
Some global settings are:</p>
<div class="highlight"><pre>ForwardAgent yes
ForwardX11 yes
</pre></div>
<p>These settings globally activate the command line options <code>-A</code> and <code>-X</code>.
Settings for specific target machines can be given:</p>
<div class="highlight"><pre>Host work
User gw
Hostname mymachine.example.com
</pre></div>
<p>This would set an alias <code>work</code> for <code>gw@mymachine.example.com</code>.</p>LaTeX Links2013-08-20T09:20:00+02:00Georg Wassentag:www.wassen.net,2013-08-20:latex-links.html<h1 id="typography">Typography</h1>
<ul>
<li><a href="http://practicaltypography.com/index.html">Book: Butterick’s Practical Typography</a></li>
</ul>
<h1 id="tools">Tools</h1>
<ul>
<li><a href="http://www.dante.de/CTAN//help/Catalogue/brief.html">CTAN Package Catalogue</a>,
a comprehensive (not to say overwhelming) list of LaTeX packages.
The detail pages tell, if the package is available in your LaTeX distribution and hold links to the documentation.
The link goes to a load balancer that redirects to a mirror.</li>
<li><a href="https://www.writelatex.com/">Writelatex.com</a>, a collaborative online editor that even works on tablets and phones.</li>
</ul>SQL plot2013-08-19T21:31:00+02:00Georg Wassentag:www.wassen.net,2013-08-19:sql-plot.html<h1 id="intro">Intro</h1>
<p>For quickly processing benchmark data, I wrote a rather complex bash script
that allows querying data from a <a href="http://www.sqlite.org">SQLite</a> database
to create <a href="http://www.gnuplot.info">gnuplot</a> graphics (2D, 3D and histograms).
In retrospect, this could probably better have become a Python program,
but at that time, I was learning Bash and was experienced with SQL
after several years of a student job programming Oracle.</p>
<div class="toc"><span class="toctitle">Contents:</span><ul>
<li><a href="#intro">Intro</a></li>
<li><a href="#example">Example</a></li>
<li><a href="#documentation">Documentation</a></li>
<li><a href="#download">Download</a></li>
</ul>
</div>
<h1 id="example">Example</h1>
<p>A quick example:
The benchmark was executed for a range of array sizes and a range of loading processes,
each for reading and writing.
It issued minimum, maximum and average values for array element access times.
It ran for a full weekend and created 6400 files.
A script processed the output files and created SQL insert statements,
that are directly piped to sqlite3 to create a database file (run time: approx. 1 minute).
Using <code>sqlplot.sh</code>, the session went approximately as follows:</p>
<div class="highlight"><pre><span class="nv">$ </span>sqlplot.sh data.db
Welcome to sqlplot 0.9 <span class="o">(</span>14.6.2013<span class="o">)</span>
Print <span class="nb">help </span>to get list of commands
sqlplot> desc data
CREATE TABLE data <span class="o">(</span>
id INTEGER PRIMARY KEY,
load_method CHAR<span class="o">(</span>2<span class="o">)</span>,
load_range INTEGER,
isol_method CHAR<span class="o">(</span> 2<span class="o">)</span>,
isol_range INTEGER,
min NUMBER,
avg NUMBER,
max NUMBER<span class="o">)</span><span class="p">;</span>
> <span class="k">select</span> load_method<span class="o">||</span><span class="s1">'-'</span><span class="o">||</span>isol_method, load_range, isol_range, max from data order by 1,2,3
load_method<span class="o">||</span><span class="s1">'-'</span><span class="o">||</span>isol_method load_range isol_range max
----------------------------- ---------- ---------- ----------
r-r <span class="m">1024</span> <span class="m">1024</span> <span class="m">220</span>
r-r <span class="m">1024</span> <span class="m">4096</span> <span class="m">200</span>
r-r <span class="m">1024</span> <span class="m">16384</span> <span class="m">200</span>
r-r <span class="m">1024</span> <span class="m">32768</span> <span class="m">220</span>
r-r <span class="m">1024</span> <span class="m">65536</span> <span class="m">176</span>
r-r <span class="m">1024</span> <span class="m">131072</span> <span class="m">200</span>
r-r <span class="m">1024</span> <span class="m">262144</span> <span class="m">268</span>
r-r <span class="m">1024</span> <span class="m">524288</span> <span class="m">180</span>
r-r <span class="m">1024</span> <span class="m">1048576</span> <span class="m">204</span>
r-r <span class="m">1024</span> <span class="m">2097152</span> <span class="m">288</span>
...
rw-rw <span class="m">536870912</span> <span class="m">536870912</span> <span class="m">1213684</span>
> splot
</pre></div>
<p>The command <code>splot</code> creates a 3D plot.
It uses the three rightmost columns as x, y and z values.
Additional columns are used as keys for different data sets (here: 'r-r' etc.).
The resulting picture looked like this:</p>
<p><img alt="Screenshot" src="static/sqlplot1.png" /></p>
<p>Some further tweaking gnuplot's settings:</p>
<div class="highlight"><pre>> <span class="nb">set </span>logscale xyz 2
> <span class="nb">set </span>ticslevel 0
> <span class="nb">set </span>xlabel <span class="s1">'load'</span>
> <span class="nb">set </span>ylabel <span class="s1">'benchmark size'</span>
> <span class="nb">set </span>title <span class="s1">'max latency'</span>
> log2tics x <span class="m">1024</span> <span class="m">4</span> 536870912
> log2tics y <span class="m">1024</span> <span class="m">4</span> 536870912
> splot
> splot max.svg
</pre></div>
<p>The last line stores a SVG image that can easily be converted to PDF,
e.g. using <a href="http://www.inkscape.org">Inkscape</a>.
(I favored SVG over PDF export, because when I wrote sqlplot, the SVG export
of gnuplot was much nicer than its PDF generation capabilities.)</p>
<p><img alt="Screenshot" src="static/sqlplot2.png" /></p>
<h1 id="documentation">Documentation</h1>
<p>The output of the <code>help</code> command:</p>
<div class="highlight"><pre><span class="n">help</span> <span class="p">[</span><span class="n">xxx</span><span class="p">]</span> <span class="o">-</span> <span class="n">help</span> <span class="p">[</span><span class="n">topic</span> <span class="n">xxx</span><span class="p">]</span>
<span class="n">exit</span> <span class="o">-</span> <span class="n">quit</span> <span class="n">program</span>
<span class="n">desc</span> <span class="o">-</span> <span class="n">list</span> <span class="n">of</span> <span class="n">tables</span> <span class="n">or</span> <span class="n">table</span> <span class="n">description</span>
<span class="n">select</span> <span class="p">...</span> <span class="o">-</span> <span class="n">SQL</span> <span class="n">select</span> <span class="n">statement</span>
<span class="n">set</span> <span class="p">...</span> <span class="o">-</span> <span class="n">set</span> <span class="n">gnuplot</span> <span class="n">options</span>
<span class="n">unset</span> <span class="p">...</span> <span class="o">-</span> <span class="n">remove</span> <span class="n">gnuplot</span> <span class="n">setting</span>
<span class="n">show</span> <span class="o">-</span> <span class="n">show</span> <span class="n">current</span> <span class="n">gnuplot</span> <span class="n">settings</span>
<span class="n">reset</span> <span class="o">-</span> <span class="n">remove</span> <span class="n">all</span> <span class="n">gnuplot</span> <span class="n">settings</span>
<span class="n">plot</span> <span class="o">-</span> <span class="n">xy</span> <span class="n">plot</span> <span class="n">of</span> <span class="n">last</span> <span class="n">query</span>
<span class="n">hist</span> <span class="o">-</span> <span class="n">histogram</span> <span class="p">(</span><span class="n">bar</span> <span class="n">plot</span><span class="p">)</span> <span class="n">of</span> <span class="n">last</span> <span class="n">query</span>
<span class="n">splot</span> <span class="o">-</span> <span class="mi">3</span><span class="n">d</span> <span class="n">plot</span> <span class="n">of</span> <span class="n">last</span> <span class="n">query</span>
<span class="n">load</span> <span class="o">-</span> <span class="n">load</span> <span class="o">*</span><span class="p">.</span><span class="n">sp</span> <span class="n">script</span>
</pre></div>
<h1 id="download">Download</h1>
<p>current version: <a href="/download/sqlplot.sh">sqlplot.sh</a></p>Beamer2013-08-18T17:30:00+02:00Georg Wassentag:www.wassen.net,2013-08-18:beamer.html<h1 id="beamer">Beamer</h1>
<p>The beamer package allows to build presentation slides as PDF.
If you're friend with LaTeX (after writing a paper or thesis),
it really easy, to create similar perfectly looking slides.</p>
<p>to be continued...</p>
<p>(until then, refer to the documentation <code>texdoc beamer</code>)</p>RWTH Beamer Theme2013-08-18T17:30:00+02:00Georg Wassentag:www.wassen.net,2013-08-18:rwth-beamer-theme.html<p>I really like the <a href="http://www.wassen.net/beamer.html">Beamer package</a> for building presentation slides.
But my university only has a Powerpoint template.
So I went ahead creating a similar looking beamer style.</p>
<table>
<thead>
<tr>
<th>Version</th>
<th>Changes</th>
</tr>
</thead>
<tbody>
<tr>
<td><a href="http://www.lfbs.rwth-aachen.de/users/global/latex/beamer-0.4.zip">0.4</a></td>
<td>2013-08-18 initial publishing</td>
</tr>
</tbody>
</table>RWTH Beamer Vorlagen2013-08-18T17:30:00+02:00Georg Wassentag:www.wassen.net,2013-08-18:rwth-beamer-theme-de.html<p>Ich erstelle meine Präsentationsfolien mit LaTeX Beamer.
Da die Uni jedoch nur Powerpoint-Vorlagen bereit stellt,
habe ich eben selber einen Beamer-Stil erstellt,
der das Aussehen der Vorlage möglichst genau nachahmt.</p>
<table>
<thead>
<tr>
<th>Version</th>
<th>Änderungen</th>
</tr>
</thead>
<tbody>
<tr>
<td><a href="http://www.lfbs.rwth-aachen.de/users/global/latex/beamer-0.4.zip">0.4</a></td>
<td>2013-08-18 Erstveröffentlichung</td>
</tr>
</tbody>
</table>http Server2013-08-16T08:23:00+02:00Georg Wassentag:www.wassen.net,2013-08-16:http-server.html<p>Just enter:</p>
<div class="highlight"><pre>python -m SimpleHTTPServer
</pre></div>
<p>This will start a server in the current directory on port 8000.
I would not expose it to the wider internet, but if you quickly want to transfer some files
to another computer (when SSH is not an option...).</p>
<p>Thanks to <a href="http://www.commandlinefu.com/commands/browse/sort-by-votes">commandlinefu.com</a> (where you'll find a ton of other useful bits).</p>TexLive on OpenSuse2013-08-16T08:23:00+02:00Georg Wassentag:www.wassen.net,2013-08-16:texlive-on-opensuse.html<p>Markus Kohm (the Author of the <a href="http://developer.berlios.de/projects/koma-script3/">KOMA-Script document classes</a>) has
created an OpenSuse RPM that provides the dependencies for other packages.
Unlike the original OpenSuse texlive package, it does not install a fixed texlive version (OpenSuse still ships 2012),
but it offers a GUI to download and maintain a TexLive installation from their repositories.
Similar to Linux distributions, TexLive is also a distribution with mirrored repositories and dependency-regarding package manager.</p>
<p>The page is in German, but here's how it works:</p>
<ul>
<li>Install the <a href="http://www.komascript.de/texlive">TexLive package</a> via 1-Click Installation (the blue button).
During this process, Yast/Zypper will ask to remove hundreds of texlive packages from its own repository.</li>
<li>Re-login.
Your account was added to the new group <code>texlive</code> which is allowed to run the tool
and the new group becomes active only after a new login (can be checked with <code>id</code>).</li>
<li>Call <code>texlive-config</code> either as normal user (to install just for yourself) or as root
(for a system-wide installation with symlinks in <code>/usr/local/bin</code>)</li>
<li>The GUI offers some settings:<ul>
<li>The installed TexLive versions are shown on the left and can be activated alternately.</li>
<li>Install a new TexLive version</li>
</ul>
</li>
<li>Before installing, more choices can be done<ul>
<li>If you're installing as normal user,
make sure that you <em>do not</em> select "install to <code>/usr/local/bin</code>",
this will fail and you have to restart the installation.</li>
<li>Otherwise, when executing as root,
you probably want to choose this setting.</li>
</ul>
</li>
<li>After installing, you either need either to re-login or to <code>source /etc/profile.d/zzz-texlive.sh</code>.
The latter command sets environment variables which must be set for every shell.
When logging in, the scripts in <code>/etc/profile.d</code> are executed and all shells inherit their settings.</li>
</ul>C Makefile2013-08-15T22:12:00+02:00Georg Wassentag:www.wassen.net,2013-08-15:c-makefile.html<p>Example Makefile:</p>
<div class="highlight"><pre><span class="nv">MAIN</span><span class="o">=</span><span class="k">$(</span>sh egrep -l <span class="s1">'\<main\>.*\('</span> *.c<span class="k">)</span>
<span class="nv">CFILES</span><span class="o">=</span><span class="k">$(</span>filter-out <span class="k">$(</span>MAIN<span class="k">)</span>, <span class="k">$(</span>sh ls *.c<span class="k">))</span>
<span class="nv">OBJS</span><span class="o">=</span><span class="k">$(</span>MAIN:.c<span class="o">=</span>.o<span class="k">)</span> <span class="k">$(</span>CFILES:.c<span class="o">=</span>.o<span class="k">)</span>
<span class="nv">EXEC</span><span class="o">=</span><span class="k">$(</span>MAIN:.c<span class="o">=</span><span class="k">)</span>
<span class="nv">CC</span><span class="o">=</span>gcc
<span class="nv">CFLAGS</span><span class="o">=</span>-g -O2
<span class="nv">LDFLAGS</span><span class="o">=</span>
<span class="nv">LDLIBS</span><span class="o">=</span>
<span class="nf">DEFAULT</span><span class="o">:</span> <span class="k">$(</span><span class="nv">EXEC</span><span class="k">)</span>
<span class="nf">$(EXEC) </span><span class="o">:</span> <span class="k">$(</span><span class="nv">OBJS</span><span class="k">)</span>
<span class="k">$(</span>CC<span class="k">)</span> <span class="k">$(</span>LDFLAGS<span class="k">)</span> -o <span class="nv">$@</span> <span class="nv">$^</span> <span class="k">$(</span>LDLIBS<span class="k">)</span>
</pre></div>
<ul>
<li>The variable <code>MAIN</code> is populated with egrep searching for a file that contains the string <code>main</code> (not the most stable algorithm...)</li>
<li><code>CFILES</code> are all other files with the file extension .c.</li>
<li><code>OBJS</code> are all C-files (from <code>MAIN</code> and <code>CFILES</code>) with the extension .c replaced with .o.</li>
<li><code>EXEC</code> will become the name of the executable. You can change this according to your preference.</li>
<li>Conventional variables used by the standard rules: <code>CC</code> is the name of the C compiler, <code>CFLAGS</code> are given for compiling,
<code>LDFLAGS</code> are given for linking and <code>LDLIBS</code> are libraries that need to be linked.</li>
<li>The first rule is the default when <code>make</code> is invoked without parameters.
To avoid confusion and accidental overwriting, a striking <code>DEFAULT</code>-target is used
that just tells <code>make</code> to build the executable.</li>
<li>The only rule is given to link the executable <code>EXEC</code> from all object files.</li>
<li>For the object files, we don't need to give a rule because <code>make</code> has standard rules for such common tasks.</li>
<li>The variables <code>CC</code>, <code>CFLAGS</code>, <code>LDFLAGS</code>, and <code>LDLIBS</code> are also used by built-in rules from GNU make (gmake)
that can be analyzed by <code>make -p</code>.</li>
</ul>LaTeX Makefile2013-08-15T22:12:00+02:00Georg Wassentag:www.wassen.net,2013-08-15:latex-makefile.html<p>Example Makefile:</p>
<div class="highlight"><pre><span class="nv">MAIN</span><span class="o">=</span><span class="k">$(</span>sh grep -l <span class="s1">'\documentclass'</span> *.tex<span class="k">)</span>
<span class="nv">TEXFILES</span><span class="o">=</span><span class="k">$(</span>filter-out <span class="k">$(</span>MAIN<span class="k">)</span>, <span class="k">$(</span>sh ls *.tex<span class="k">))</span>
<span class="nv">PDFLATEX</span><span class="o">=</span>pdflatex
<span class="nv">PDF</span><span class="o">=</span><span class="k">$(</span>MAIN:.tex<span class="o">=</span>.pdf<span class="k">)</span>
<span class="nf">DEFAULT</span><span class="o">:</span> <span class="n">pdf</span>
<span class="nf">pdf </span><span class="o">:</span> <span class="k">$(</span><span class="nv">PDF</span><span class="k">)</span>
<span class="nf">$(PDF) </span><span class="o">:</span> <span class="k">$(</span><span class="nv">MAIN</span><span class="k">)</span> <span class="k">$(</span><span class="nv">TEXFILES</span><span class="k">)</span>
<span class="k">$(</span>PDFLATEX<span class="k">)</span> <span class="k">$(</span>MAIN<span class="k">)</span>
<span class="k">$(</span>PDFLATEX<span class="k">)</span> <span class="k">$(</span>MAIN<span class="k">)</span>
</pre></div>
<ul>
<li>The variable <code>MAIN</code> is automatically populated with grep searching for a file that contains the string <code>\documentclass</code> (hopefully finds the correct one...)</li>
<li><code>TEXFILES</code> are all other files with the file extension .tex.</li>
<li><code>PDF</code> will become the name of the generated PDF. It is taken from the <code>MAIN</code> file. You can change this according to your preference.</li>
<li><code>PDFLATEX</code> is the program name to generate PDF from LaTeX input.</li>
<li>The first rule is the default when <code>make</code> is invoked without parameters.
To avoid confusion and accidental overwriting, a striking <code>DEFAULT</code>-target is used
that just tells <code>make</code> to build the PDF.</li>
<li>The rule for <code>make pdf</code> is propably only useful, if also rules for Postscript and DVI are implemented.
For beginners: stick to using <code>pdflatex</code> to directly generate PDF files.
The old way with DVI is obsolete (unless your printing service can only handle Postscript).</li>
<li>The PDF is generated from all .tex files.
To get the references right, <code>pdflatex</code> is called twice.</li>
<li>If you're using an index or BibLaTeX/BibTeX, then append their invokation and add another two calls to <code>pdflatex</code>.</li>
<li>
<p>If the generation takes too long, add a <code>quick</code> target, that unconditionally calls <code>pdflatex</code> once.
After fixing typos or only minor modifications, this often suffices.</p>
<p>::make
quick :
$(PDFLATEX) $(MAIN)</p>
</li>
</ul>Important C Libraries2013-08-15T21:14:00+02:00Georg Wassentag:www.wassen.net,2013-08-15:important-c-libraries.html<p>If you're using these libraries for your projects, you propably need
to install the <code>-devel</code> versions from your distribution's repository.</p>
<div class="toc"><span class="toctitle">Contents:</span><ul>
<li><a href="#libc-the-standard-c-library">libc - the Standard C Library</a><ul>
<li><a href="#math">math</a></li>
<li><a href="#posix-threads">POSIX threads</a></li>
</ul>
</li>
<li><a href="#gnome-universe">Gnome universe</a><ul>
<li><a href="#glib">glib</a></li>
<li><a href="#gtk">GTK+</a></li>
</ul>
</li>
<li><a href="#gnu-scientific-library">GNU Scientific Library</a></li>
</ul>
</div>
<h1 id="libc-the-standard-c-library">libc - the Standard C Library</h1>
<p>The C library is part of the C standard and contains functions such as <code>printf()</code> and <code>malloc()</code>.
It is linked with <code>-lc</code>, but that's default and you don't need to provide this parameter.</p>
<p>The most important (okay, for open source programmers) implementation is the <a href="http://www.gnu.org/software/libc">GNU libc</a> (glibc).
Don't confuse the glibc with the glib (described further below).</p>
<h2 id="math">math</h2>
<p>The math library contains functions more advanced than a simple calculator (think scientific calculator,
but see also the GNU Scientific Library, further below).</p>
<p>Unlike the standard C library, the math part must be linked explicitly with <code>-lm</code>.</p>
<h2 id="posix-threads">POSIX threads</h2>
<p>Threads are multiple paths of execution inside a process that share its address space.
Long running tasks can be delegated to threads to avoid blocking the main process.
On today's multi-processor systems, they can be executed concurrently to speed up things.
But in any case, the synchronization, especially for access to shared resources, must be taken care of manually.</p>
<p>Link with <code>-pthread</code> or <code>-lpthread</code>.</p>
<h1 id="gnome-universe">Gnome universe</h1>
<h2 id="glib">glib</h2>
<p>This is a part of GTK+ providing basic functionality Gnome is based on.</p>
<p>gobject?</p>
<h2 id="gtk">GTK+</h2>
<p>The object-oriented widget library, that makes the Gnome GUI and many other programs.</p>
<h1 id="gnu-scientific-library">GNU Scientific Library</h1>
<p>t.b.d.</p>