mitya57’s weblog - Planethttps://mitya57.me/weblog/2021-02-21T21:30:00+03:00ReText turns 10 years2021-02-21T21:30:00+03:002021-02-21T21:30:00+03:00Dmitry Shachnevtag:mitya57.me,2021-02-21:/weblog/2021/02/retext-turns-10-years.html<p>Exactly ten years ago, in February 2011, the <a href="https://github.com/retext-project/retext/commit/f278a6c91a5660f5">first commit</a> in ReText git
repository was made. It was just a single 364 lines Python file back then (now
the project has more than 6000 lines of Python code).</p>
<p>Since 2011, the editor migrated from SourceForge to GitHub, gained a lot …</p><p>Exactly ten years ago, in February 2011, the <a href="https://github.com/retext-project/retext/commit/f278a6c91a5660f5">first commit</a> in ReText git
repository was made. It was just a single 364 lines Python file back then (now
the project has more than 6000 lines of Python code).</p>
<p>Since 2011, the editor migrated from SourceForge to GitHub, gained a lot of new
features, and — most importantly — now there is an active community around it,
which includes both long-time contributors and newcomers who create their first
issues or pull requests. I don’t always have enough time to reply to issues
or implement new features myself, but the community members help me with this.</p>
<p>Earlier this month, I made a new release (7.2), which adds a side panel with
directory tree (contributed by <a href="https://github.com/xgouchet">Xavier Gouchet</a>), option to fully highlight
wrapped lines (contributed by <a href="https://github.com/nihillum">nihillum</a>), ability to search in the preview
mode and much more — see the <a href="https://github.com/retext-project/retext/releases/tag/7.2.0">release page</a> on GitHub.</p>
<p><img alt="Side panel in ReText" src="/static/side-panel.png"></p>
<p>Also a new version of <a href="https://github.com/retext-project/pymarkups">PyMarkups</a> module was released, which contains all the
code for processing various markup languages. It now supports
<a href="https://pymarkups.readthedocs.io/en/latest/standard_markups.html#markdown-markup">markdown-extensions.yaml</a> files which allow specifying complex <a href="https://github.com/retext-project/retext/wiki/Markdown-extensions">extensions</a>
options and adds initial support for <a href="http://docs.mathjax.org/en/latest/upgrading/whats-new-3.0.html">MathJax 3</a>.</p>
<p>Also check out the <a href="https://github.com/retext-project/retext/releases/tag/7.1.0">release notes for 7.1</a> which was not announced on this
blog.</p>
<p>Future plans include making at least one more release this year, adding support
for <a href="https://www.qt.io/blog/qt-6.0-released">Qt 6</a>. Qt 5 support will last for at least one more year.</p>A review of endianness bugs in Qt, and how they were fixed2021-01-01T12:35:00+03:002021-01-01T12:35:00+03:00Dmitry Shachnevtag:mitya57.me,2021-01-01:/weblog/2021/01/qt-big-endian-history.html<p>As you may know, I am Qt 5 maintainer in Debian. Maintaning Qt means not only
bumping the version each time a new version is released, but also making sure
Qt builds successfully on all architectures that are supported in Debian (and
for some submodules, the automatic tests pass).</p>
<p>An …</p><p>As you may know, I am Qt 5 maintainer in Debian. Maintaning Qt means not only
bumping the version each time a new version is released, but also making sure
Qt builds successfully on all architectures that are supported in Debian (and
for some submodules, the automatic tests pass).</p>
<p>An important sort of build failures are endianness specific failures. Most
widely used architectures (x86_64, aarch64) are little endian. However, Debian
officially supports one big endian architecture (<a href="https://en.wikipedia.org/wiki/Linux_on_IBM_Z">s390x</a>), and unofficially
a few more ports are provided, such as <a href="https://wiki.debian.org/PPC64">ppc64</a> and <a href="https://wiki.debian.org/Sparc64">sparc64</a>.</p>
<p>Unfortunately, Qt upstream does not have any big endian machine in their CI
system, so endianness issues get noticed only when the packages fail to build
on our build daemons. In the last years I have discovered and fixed some such
issues in various parts of Qt, so I decided to write a post to illustrate how
to write really cross-platform C/C++ code.</p>
<h2>Issue 1: the WebP image format handler (<a href="https://codereview.qt-project.org/c/qt/qtimageformats/+/86275">code review</a>)</h2>
<p>The relevant code snippet is:</p>
<div class="highlight"><pre><span></span><code><span class="k">if</span> <span class="p">(</span><span class="n">srcImage</span><span class="p">.</span><span class="n">format</span><span class="p">()</span> <span class="o">!=</span> <span class="n">QImage</span><span class="o">::</span><span class="n">Format_ARGB32</span><span class="p">)</span>
<span class="n">srcImage</span> <span class="o">=</span> <span class="n">srcImage</span><span class="p">.</span><span class="n">convertToFormat</span><span class="p">(</span><span class="n">QImage</span><span class="o">::</span><span class="n">Format_ARGB32</span><span class="p">);</span>
<span class="c1">// ...</span>
<span class="k">if</span> <span class="p">(</span><span class="o">!</span><span class="n">WebPPictureImportBGRA</span><span class="p">(</span><span class="o">&</span><span class="n">picture</span><span class="p">,</span> <span class="n">srcImage</span><span class="p">.</span><span class="n">bits</span><span class="p">(),</span> <span class="n">srcImage</span><span class="p">.</span><span class="n">bytesPerLine</span><span class="p">()))</span> <span class="p">{</span>
<span class="c1">// ...</span>
<span class="p">}</span>
</code></pre></div>
<p>The code here is serializing the images into <code>QImage::Format_ARGB32</code> format, and
then passing the bytes into WebP’s import function. With this format, the image
is stored using a 32-bit ARGB format (0xAARRGGBB). This means that the bytes
will be 0xBB, 0xGG, 0xRR, 0xAA or little endian and 0xAA, 0xRR, 0xGG, 0xBB on
big endian. However, <code>WebPPictureImportBGRA</code> expects the first format on all
architectures.</p>
<p>The fix was to use <code>QImage::Format_RGBA8888</code>. As the <a href="https://doc.qt.io/qt-5/qimage.html#Format-enum">QImage documentation</a>
says, with this format <em>the order of the colors is the same on any architecture
if read as bytes 0xRR, 0xGG, 0xBB, 0xAA</em>.</p>
<h2>Issue 2: <code>qimage_converter_map</code> structure (<a href="https://codereview.qt-project.org/c/qt/qtbase/+/101769">code review</a>)</h2>
<p>The code seems to already support big endian. But maybe you can spot the error?</p>
<div class="highlight"><pre><span></span><code><span class="cp">#if Q_BYTE_ORDER == Q_LITTLE_ENDIAN</span>
<span class="mi">0</span><span class="p">,</span>
<span class="n">convert_ARGB_to_ARGB_PM</span><span class="p">,</span>
<span class="cp">#else</span>
<span class="mi">0</span><span class="p">,</span>
<span class="mi">0</span>
<span class="cp">#endif</span>
</code></pre></div>
<p>It is the missing comma! It is present in the little endian block, but not in
the big endian one. This was fixed trivially.</p>
<h2>Issue 3: QHandle, part of Qt 3D module (<a href="https://codereview.qt-project.org/c/qt/qt3d/+/162737">code review</a>)</h2>
<p>QHandle class uses a union that is declared as follows:</p>
<div class="highlight"><pre><span></span><code><span class="k">struct</span> <span class="nc">Data</span> <span class="p">{</span>
<span class="n">quint32</span> <span class="nl">m_index</span> <span class="p">:</span> <span class="n">IndexBits</span><span class="p">;</span>
<span class="n">quint32</span> <span class="nl">m_counter</span> <span class="p">:</span> <span class="n">CounterBits</span><span class="p">;</span>
<span class="n">quint32</span> <span class="nl">m_unused</span> <span class="p">:</span> <span class="mi">2</span><span class="p">;</span>
<span class="p">};</span>
<span class="k">union</span> <span class="p">{</span>
<span class="n">Data</span> <span class="n">d</span><span class="p">;</span>
<span class="n">quint32</span> <span class="n">m_handle</span><span class="p">;</span>
<span class="p">};</span>
</code></pre></div>
<p>The sizes are declared such as IndexBits + CounterBits + 2 is always
equal to 32 (four bytes).</p>
<p>Then we have a constructor that sets the values of Data struct:</p>
<div class="highlight"><pre><span></span><code><span class="n">QHandle</span><span class="p">(</span><span class="n">quint32</span> <span class="n">i</span><span class="p">,</span> <span class="n">quint32</span> <span class="n">count</span><span class="p">)</span>
<span class="p">{</span>
<span class="n">d</span><span class="p">.</span><span class="n">m_index</span> <span class="o">=</span> <span class="n">i</span><span class="p">;</span>
<span class="n">d</span><span class="p">.</span><span class="n">m_counter</span> <span class="o">=</span> <span class="n">count</span><span class="p">;</span>
<span class="n">d</span><span class="p">.</span><span class="n">m_unused</span> <span class="o">=</span> <span class="mi">0</span><span class="p">;</span>
<span class="p">}</span>
</code></pre></div>
<p>The value of <code>m_handle</code> will be different depending on endianness!
So the test that was expecting a particular value with given constructor
arguments was failing. I fixed it by using the following macro:</p>
<div class="highlight"><pre><span></span><code><span class="cp">#if Q_BYTE_ORDER == Q_BIG_ENDIAN</span>
<span class="cp">#define GET_EXPECTED_HANDLE(qHandle) ((qHandle.index() << (qHandle.CounterBits + 2)) + (qHandle.counter() << 2))</span>
<span class="cp">#else </span><span class="cm">/* Q_LITTLE_ENDIAN */</span><span class="cp"></span>
<span class="cp">#define GET_EXPECTED_HANDLE(qHandle) (qHandle.index() + (qHandle.counter() << qHandle.IndexBits))</span>
<span class="cp">#endif</span>
</code></pre></div>
<h2>Issue 4: QML compiler (<a href="https://codereview.qt-project.org/c/qt/qtdeclarative/+/197746">code review</a>)</h2>
<p>The QML compiler used a helper class named LEUInt32 (based on <a href="https://doc.qt.io/qt-5/qleinteger.html">QLEInteger</a>)
that always stored the numbers in little endian internally. This class
can be safely mixed with native quint32 on little endian systems, but not
on big endian.</p>
<p>Usually the compiler would warn about type mismatch, but here the code used
<code>reinterpret_cast</code>, such as:</p>
<div class="highlight"><pre><span></span><code><span class="n">quint32</span> <span class="o">*</span><span class="n">objectTable</span> <span class="o">=</span> <span class="k">reinterpret_cast</span><span class="o"><</span><span class="n">quint32</span><span class="o">*></span><span class="p">(</span><span class="n">data</span> <span class="o">+</span> <span class="n">qmlUnit</span><span class="o">-></span><span class="n">offsetToObjects</span><span class="p">);</span>
</code></pre></div>
<p>So this was not noticed on build time, but the compiler was crashing. The fix
was trivial again, replacing quint32 with QLEUInt32.</p>
<h2>Issue 5: QModbusPdu, part of Qt Serial Bus module (<a href="https://codereview.qt-project.org/c/qt/qtserialbus/+/245206">code review</a>)</h2>
<p>The code snippet is simple:</p>
<div class="highlight"><pre><span></span><code><span class="n">QModbusPdu</span><span class="o">::</span><span class="n">FunctionCode</span> <span class="n">code</span> <span class="o">=</span> <span class="n">QModbusPdu</span><span class="o">::</span><span class="n">Invalid</span><span class="p">;</span>
<span class="k">if</span> <span class="p">(</span><span class="n">stream</span><span class="p">.</span><span class="n">readRawData</span><span class="p">((</span><span class="kt">char</span> <span class="o">*</span><span class="p">)</span> <span class="p">(</span><span class="o">&</span><span class="n">code</span><span class="p">),</span> <span class="k">sizeof</span><span class="p">(</span><span class="n">quint8</span><span class="p">))</span> <span class="o">!=</span> <span class="k">sizeof</span><span class="p">(</span><span class="n">quint8</span><span class="p">))</span>
<span class="k">return</span> <span class="n">stream</span><span class="p">;</span>
</code></pre></div>
<p><code>QModbusPdu::FunctionCode</code> is an enum, so <code>code</code> is a multi-byte value (even if
only one byte is significant). However, <code>(char *) (&code)</code> returns a pointer to
the first byte of it. It is the needed byte on little endian systems, but it is
the wrong byte on big endian ones!</p>
<p>The correct fix was using a temporary one-byte variable:</p>
<div class="highlight"><pre><span></span><code><span class="n">quint8</span> <span class="n">codeByte</span> <span class="o">=</span> <span class="mi">0</span><span class="p">;</span>
<span class="k">if</span> <span class="p">(</span><span class="n">stream</span><span class="p">.</span><span class="n">readRawData</span><span class="p">((</span><span class="kt">char</span> <span class="o">*</span><span class="p">)</span> <span class="p">(</span><span class="o">&</span><span class="n">codeByte</span><span class="p">),</span> <span class="k">sizeof</span><span class="p">(</span><span class="n">quint8</span><span class="p">))</span> <span class="o">!=</span> <span class="k">sizeof</span><span class="p">(</span><span class="n">quint8</span><span class="p">))</span>
<span class="k">return</span> <span class="n">stream</span><span class="p">;</span>
<span class="n">QModbusPdu</span><span class="o">::</span><span class="n">FunctionCode</span> <span class="n">code</span> <span class="o">=</span> <span class="p">(</span><span class="n">QModbusPdu</span><span class="o">::</span><span class="n">FunctionCode</span><span class="p">)</span> <span class="n">codeByte</span><span class="p">;</span>
</code></pre></div>
<h2>Issue 6: <code>qt_is_ascii</code> (<a href="https://codereview.qt-project.org/c/qt/qtbase/+/258164">code review</a>)</h2>
<p>This function, as the name says, checks whether a string is ASCII. It does
that by splitting the string into 4-byte chunks:</p>
<div class="highlight"><pre><span></span><code><span class="k">while</span> <span class="p">(</span><span class="n">ptr</span> <span class="o">+</span> <span class="mi">4</span> <span class="o"><=</span> <span class="n">end</span><span class="p">)</span> <span class="p">{</span>
<span class="n">quint32</span> <span class="n">data</span> <span class="o">=</span> <span class="n">qFromUnaligned</span><span class="o"><</span><span class="n">quint32</span><span class="o">></span><span class="p">(</span><span class="n">ptr</span><span class="p">);</span>
<span class="k">if</span> <span class="p">(</span><span class="n">data</span> <span class="o">&=</span> <span class="mh">0x80808080U</span><span class="p">)</span> <span class="p">{</span>
<span class="n">uint</span> <span class="n">idx</span> <span class="o">=</span> <span class="n">qCountTrailingZeroBits</span><span class="p">(</span><span class="n">data</span><span class="p">);</span>
<span class="n">ptr</span> <span class="o">+=</span> <span class="n">idx</span> <span class="o">/</span> <span class="mi">8</span><span class="p">;</span>
<span class="k">return</span> <span class="nb">false</span><span class="p">;</span>
<span class="p">}</span>
<span class="n">ptr</span> <span class="o">+=</span> <span class="mi">4</span><span class="p">;</span>
<span class="p">}</span>
</code></pre></div>
<p><code>idx / 8</code> is the number of trailing zero bytes. However, the bytes which are
trailing on little endian are actually leading on big endian! So we can use
<code>qCountLeadingZeroBits</code> there.</p>
<h2>Issue 7: the bundled copy of tinycbor (<a href="https://github.com/thiagomacieira/tinycbor/pull/1">upstream pull request</a>)</h2>
<p>Similar to issue 5, the code was reading into the wrong byte:</p>
<div class="highlight"><pre><span></span><code><span class="k">if</span> <span class="p">(</span><span class="n">bytesNeeded</span> <span class="o"><=</span> <span class="mi">2</span><span class="p">)</span> <span class="p">{</span>
<span class="n">read_bytes_unchecked</span><span class="p">(</span><span class="n">it</span><span class="p">,</span> <span class="o">&</span><span class="n">it</span><span class="o">-></span><span class="n">extra</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="n">bytesNeeded</span><span class="p">);</span>
<span class="k">if</span> <span class="p">(</span><span class="n">bytesNeeded</span> <span class="o">==</span> <span class="mi">2</span><span class="p">)</span>
<span class="n">it</span><span class="o">-></span><span class="n">extra</span> <span class="o">=</span> <span class="n">cbor_ntohs</span><span class="p">(</span><span class="n">it</span><span class="o">-></span><span class="n">extra</span><span class="p">);</span>
<span class="p">}</span>
</code></pre></div>
<p><code>extra</code> has type uint16_t, so it has two bytes. When we need only one byte,
we read into the wrong byte, so the resulting number is 256 times higher on
big endian than it should be. Adding a temporary one-byte variable fixed it.</p>
<h2>Issue 8: perfparser, part of Qt Creator (<a href="https://codereview.qt-project.org/c/qt-creator/perfparser/+/305990">code review</a>)</h2>
<p>Here it is not trivial to find the issue just looking at the code:</p>
<div class="highlight"><pre><span></span><code><span class="n">qint32</span> <span class="n">dataStreamVersion</span> <span class="o">=</span> <span class="n">qToLittleEndian</span><span class="p">(</span><span class="n">QDataStream</span><span class="o">::</span><span class="n">Qt_DefaultCompiledVersion</span><span class="p">);</span>
</code></pre></div>
<p>However the linker was producing an error:</p>
<blockquote>
<p>undefined reference to `QDataStream::Version qbswap<QDataStream::Version>(QDataStream::Version)'</p>
</blockquote>
<p>On little endian systems, <code>qToLittleEndian</code> is a no-op, but on big endian
systems, it is a template function defined for some known types. But it turns
out we need to explicitly convert enum values to a simple type, so the fix
was passing <code>qint32(QDataStream::Qt_DefaultCompiledVersion)</code> to that function.</p>
<h2>Issue 9: Qt Personal Information Management (<a href="https://codereview.qt-project.org/c/qt/qtpim/+/319865">code review</a>)</h2>
<p>The code in test was trying to represent a number as a sequence of bytes,
using <code>reinterpret_cast</code>:</p>
<div class="highlight"><pre><span></span><code><span class="k">static</span> <span class="kr">inline</span> <span class="n">QContactId</span> <span class="n">makeId</span><span class="p">(</span><span class="k">const</span> <span class="n">QString</span> <span class="o">&</span><span class="n">managerName</span><span class="p">,</span> <span class="n">uint</span> <span class="n">id</span><span class="p">)</span>
<span class="p">{</span>
<span class="k">return</span> <span class="nf">QContactId</span><span class="p">(</span><span class="n">QStringLiteral</span><span class="p">(</span><span class="s">"qtcontacts:basic%1:"</span><span class="p">).</span><span class="n">arg</span><span class="p">(</span><span class="n">managerName</span><span class="p">),</span> <span class="n">QByteArray</span><span class="p">(</span><span class="k">reinterpret_cast</span><span class="o"><</span><span class="k">const</span> <span class="kt">char</span> <span class="o">*></span><span class="p">(</span><span class="o">&</span><span class="n">id</span><span class="p">),</span> <span class="k">sizeof</span><span class="p">(</span><span class="n">uint</span><span class="p">)));</span>
<span class="p">}</span>
</code></pre></div>
<p>The order of bytes will be different on little endian and big endian systems!
The fix was adding this line to the beginning of the function:</p>
<div class="highlight"><pre><span></span><code><span class="n">id</span> <span class="o">=</span> <span class="n">qToLittleEndian</span><span class="p">(</span><span class="n">id</span><span class="p">);</span>
</code></pre></div>
<p>This will cause the bytes to be reversed on big endian systems.</p>
<h2>What remains unfixed</h2>
<p>There are still some bugs, which require deeper investigation, for example:</p>
<ul>
<li><a href="https://bugreports.qt.io/browse/QTBUG-56806">https://bugreports.qt.io/browse/QTBUG-56806</a></li>
<li><a href="https://bugreports.qt.io/browse/QTBUG-56975">https://bugreports.qt.io/browse/QTBUG-56975</a></li>
</ul>
<p>P.S. We are <a href="https://perezmeyer.blogspot.com/2020/08/stepping-down-as-qt-6-maintainers.html">looking for new people</a> to help with maintaining Qt 6.
Join our team if you want to do some fun work like described above!</p>Qt packages built with OpenGL ES support are now available2020-01-15T17:55:00+03:002020-01-15T17:55:00+03:00Dmitry Shachnevtag:mitya57.me,2020-01-15:/weblog/2020/01/qt-opengl-es-packages-available.html<p>Some time ago, there was a <a href="https://lists.debian.org/debian-devel/2018/11/msg00457.html">thread</a> on debian-devel where we discussed how to
make Qt packages work on hardware that supports <a href="https://www.khronos.org/opengles/">OpenGL ES</a>, but not the
desktop OpenGL.</p>
<p>My first proposal was to switch to OpenGL ES by default on ARM64, as that is
the main affected architecture. After …</p><p>Some time ago, there was a <a href="https://lists.debian.org/debian-devel/2018/11/msg00457.html">thread</a> on debian-devel where we discussed how to
make Qt packages work on hardware that supports <a href="https://www.khronos.org/opengles/">OpenGL ES</a>, but not the
desktop OpenGL.</p>
<p>My first proposal was to switch to OpenGL ES by default on ARM64, as that is
the main affected architecture. After a lengthy discussion, it was decided to
ship two versions of Qt packages instead, to support more (OpenGL variant,
architecture) configurations.</p>
<p>So now I am announcing that we finally have the versions of <a href="https://packages.debian.org/sid/libqt5gui5-gles">Qt GUI</a> and
<a href="https://packages.debian.org/sid/libqt5quick5-gles">Qt Quick</a> libraries that are <em>built against OpenGL ES</em>, and the release
team helped us to rebuild the archive for compatibility with them.
These packages are <em>not co-installable</em> together with the regular
(desktop OpenGL) Qt packages, as they provide the same set of shared
libraries. So most packages now have an alternative dependency like
<code>libqt5gui5 (>= 5.x) | libqt5gui5-gles (>= 5.x)</code>. Packages get such a
dependency automatically if they are using <code>${shlibs:Depends}</code>.</p>
<p>These Qt packages will be mostly needed by ARM64 users, however they may
be also useful on other architectures too. Note that <em>armel and armhf are
not affected</em>, because there Qt was built against OpenGL ES from the very
beginning. So far there are no plans to make two versions of Qt on these
architectures, however we are open to bug reports.</p>
<p>To try that on your system (running Bullseye or Sid), just run this command:</p>
<div class="highlight"><pre><span></span><code><span class="err"># apt install libqt5gui5-gles libqt5quick5-gles</span>
</code></pre></div>
<p>The other Qt submodule packages do not need a second variant, because
they do not use any OpenGL API directly. Most of the Qt applications are
installable with these packages. At the moment, Plasma is not installable
because <a href="https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=948908">plasma-desktop FTBFS</a>, but that will be fixed sooner or later.</p>
<p>One major missing thing is PyQt5. It is linking against some Qt helper
functions that only exist for desktop OpenGL build, so we will probably
need to <a href="https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=948112#15">build a special version of PyQt5</a> for OpenGL ES.</p>
<p>If you want to use any OpenGL ES specific API in your package, build it
against qtbase5-gles-dev package instead of qtbase5-dev. There is no
qtdeclarative5-gles-dev so far, however if you need it, please let us know.</p>
<p>In case you have any questions, please feel free to file a bug against
one of the new packages, or contact us at the <a href="mailto:pkg-kde-talk@alioth-lists.debian.net">pkg-kde-talk</a> mailing list.</p>ReText 6.0 and PyMarkups 2.0 released2016-05-10T14:45:00+03:002016-05-10T14:45:00+03:00Dmitry Shachnevtag:mitya57.me,2016-05-10:/weblog/2016/05/retext-60-released.html<p>Today I have released the new major version of the <a href="https://github.com/retext-project/retext">ReText</a> editor.</p>
<p>This release would not be possible without <a href="https://github.com/Griffon26">Maurice van der Pot</a> who was the
author of the greatest features because of which the version number was bumped:</p>
<ul>
<li>
<p><strong>Synchronized scrolling</strong> for Markdown. It means that the preview area
scrolls …</p></li></ul><p>Today I have released the new major version of the <a href="https://github.com/retext-project/retext">ReText</a> editor.</p>
<p>This release would not be possible without <a href="https://github.com/Griffon26">Maurice van der Pot</a> who was the
author of the greatest features because of which the version number was bumped:</p>
<ul>
<li>
<p><strong>Synchronized scrolling</strong> for Markdown. It means that the preview area
scrolls automatically to make sure the paragraph you are editing is visible.</p>
</li>
<li>
<p>The <strong>multi-process mode</strong>. The text conversion now happens in a separate
process so will never block the UI. As a result, the interface should be
now more responsive when editing huge texts.</p>
<p>This also required refactoring the <a href="https://github.com/retext-project/pymarkups">PyMarkups</a> library. ReText now depends
on version 2.0 of it, which was released yesterday (or a newer version).</p>
</li>
</ul>
<p>Other news worth mentioning are:</p>
<ul>
<li>
<p>The tags box was replaced with a new “Formatting” box with quick access to
most Markdown features, such as headers, lists, links, images and so on.
The initial version of this code was contributed by <a href="https://github.com/dmarrazzo">Donato Marrazzo</a>.</p>
</li>
<li>
<p>Images now can be pasted into ReText: first a save dialog for them will be
presented, and then the valid Markdown or reStructuredText markup for them
will be inserted. This was contributed by Bart Clephas and Maurice van der
Pot.</p>
</li>
<li>
<p>The basic CSS styling for tables has been added (to make sure they have
borders visible). This applies only unless you have your own CSS.</p>
</li>
<li>
<p>Added a button to quickly close the search bar, for those who prefer buttons
over keyboard shortcuts.</p>
</li>
<li>
<p>Hitting <kbd>Return</kbd> key twice now will close the itemized lists and
remove the inserted list markers from the previous line.</p>
</li>
</ul>
<p>As usual, some bugs have been fixed (most of fixes have also been backported to
5.3.1 release), and some translations have been updated from <a href="https://www.transifex.com/mitya57/ReText/">Transifex</a>.</p>
<p>Please report any bugs you find to our <a href="https://github.com/retext-project/retext/issues">issue tracker</a>.</p>ReText 5.3 released2015-12-22T14:00:00+03:002015-12-22T14:00:00+03:00Dmitry Shachnevtag:mitya57.me,2015-12-22:/weblog/2015/12/retext-53-released.html<p>On Sunday I have released <a href="https://github.com/retext-project/retext">ReText</a> 5.3, and here finally comes the official
announcement.</p>
<p>Highlights in this release are:</p>
<ul>
<li>
<p>A code refactoring has been performed — a new “Tab” class has been added,
and all methods that affect only one tab (not the whole window) have been
moved there.</p>
<p>From …</p></li></ul><p>On Sunday I have released <a href="https://github.com/retext-project/retext">ReText</a> 5.3, and here finally comes the official
announcement.</p>
<p>Highlights in this release are:</p>
<ul>
<li>
<p>A code refactoring has been performed — a new “Tab” class has been added,
and all methods that affect only one tab (not the whole window) have been
moved there.</p>
<p>From the user’s point of view this means two things:</p>
<ul>
<li>
<p>The tabs are now draggable and reorderable (this was <a href="https://github.com/retext-project/retext/issues/127">a feature
requested long time ago</a>).</p>
</li>
<li>
<p>Some operations are now faster and more efficient. For example, in the
previous release turning the WebKit renderer on/off required removing
all the tabs and then re-adding them back, this giant hack has been
dropped now.</p>
</li>
</ul>
</li>
<li>
<p>A new previewer feature was contributed by <a href="https://github.com/Tamriel">Jan Korte</a>: now, if the document
contains a local link like</p>
<div class="highlight"><pre><span></span><code><span class="p">[</span><span class="n">click</span> <span class="n">me</span><span class="p">](</span><span class="n">foo</span><span class="p">.</span><span class="n">mkd</span><span class="p">)</span>
</code></pre></div>
<p>and a file named <code>foo.mkd</code> exists, it is opened in a new ReText tab.</p>
<p>It is also possible to specify names without the extension (just <code>foo</code>) or
relative paths (<code>../foo/bar.mkd</code>).</p>
</li>
<li>
<p>The colors used in the editor are now fully configurable via the standard
configuration mechanism. This is most useful for users of dark themes.</p>
<p>For example, you can change the color of line numbers area, the cursor
position box, and all colors used by the highlighter.</p>
<p>The possible colors and the procedure to change them is described in the
<a href="https://github.com/retext-project/retext/blob/master/configuration.md#color-scheme-setting">“Color scheme setting” section in the documentation</a>.</p>
</li>
<li>
<p>The “Display right margin at column” feature now displays the line more
precisely: in the previous version it was some pixels left to the cursor,
now it is exactly on the same horizontal position as the cursor.</p>
</li>
<li>
<p>Some bug fixes have been made for users that install ReText using <code>pip</code> or
<code>setup.py install</code>:</p>
<ul>
<li>
<p>The desktop file no longer hardcodes the path to executable in the
<code>Exec</code> field, it uses just <code>retext</code> now. This fix has been contributed
by <a href="https://github.com/Lin-Buo-Ren">Buo-Ren Lin</a>.</p>
</li>
<li>
<p>The <code>setup.py</code> script now installs the application logo into a location
where ReText can find it. Note: this works only for installs into user’s
home directory (with <code>--user</code> passed to <code>pip</code> or <code>setup.py install</code>),
installing software globally this way is <a href="https://github.com/pypa/pip/issues/1668">not recommended</a> anyway.</p>
</li>
</ul>
</li>
<li>
<p>The AppStream metadata included in the previous version was updated to fix
some warnings from the <a href="https://appstream.debian.org/">appstream.debian.org</a> metadata validator.</p>
</li>
</ul>
<p>Also, a week before ReText 5.3 a <a href="https://pymarkups.readthedocs.io/en/latest/changelog.html#version-1-0-2015-12-13">new version of PyMarkups</a> was
released, bringing enhanced support for the <a href="https://en.wikipedia.org/wiki/Textile_(markup_language)">Textile</a> markup. You can now edit
Textile files in ReText too, provided that <a href="https://github.com/textile/python-textile">python-textile module</a> for Python 3
is installed.</p>
<p>As usual, you can get the latest release from <a href="https://pypi.org/project/ReText/">PyPI</a> or from the Debian/Ubuntu
repositories.</p>
<p>Please report any bugs you find to our <a href="https://github.com/retext-project/retext/issues">issue tracker</a>.</p>Magic infinite sequences for Python2015-12-10T21:00:00+03:002015-12-10T21:00:00+03:00Dmitry Shachnevtag:mitya57.me,2015-12-10:/weblog/2015/12/infinite-sequences.html<p>Today I have released a small module for Python 3 that implements cached lazy
infinite sequences.</p>
<p>Here are some examples that demonstrate what this module can do:</p>
<div class="highlight"><pre><span></span><code><span class="gp">>>> </span><span class="n">InfSequence</span><span class="p">(</span><span class="mi">5</span><span class="p">,</span> <span class="mi">6</span><span class="p">,</span> <span class="o">...</span><span class="p">)</span>
<span class="go"><InfSequence: 5 6 7 8 9 10 ...></span>
<span class="gp">>>> </span><span class="n">InfSequence</span><span class="o">.</span><span class="n">geometric_progression</span><span class="p">(</span><span class="mi">9</span><span class="p">)</span>
<span class="go"><InfSequence: 1 9 81 729 6561 59049 ...></span>
<span class="gp">>>> </span><span class="n">InfSequence</span><span class="o">.</span><span class="n">cycle</span><span class="p">(</span><span class="s1">'foo …</span></code></pre></div><p>Today I have released a small module for Python 3 that implements cached lazy
infinite sequences.</p>
<p>Here are some examples that demonstrate what this module can do:</p>
<div class="highlight"><pre><span></span><code><span class="gp">>>> </span><span class="n">InfSequence</span><span class="p">(</span><span class="mi">5</span><span class="p">,</span> <span class="mi">6</span><span class="p">,</span> <span class="o">...</span><span class="p">)</span>
<span class="go"><InfSequence: 5 6 7 8 9 10 ...></span>
<span class="gp">>>> </span><span class="n">InfSequence</span><span class="o">.</span><span class="n">geometric_progression</span><span class="p">(</span><span class="mi">9</span><span class="p">)</span>
<span class="go"><InfSequence: 1 9 81 729 6561 59049 ...></span>
<span class="gp">>>> </span><span class="n">InfSequence</span><span class="o">.</span><span class="n">cycle</span><span class="p">(</span><span class="s1">'foo'</span><span class="p">,</span> <span class="s1">'bar'</span><span class="p">)</span>
<span class="go"><InfSequence: 'foo' 'bar' 'foo' 'bar' 'foo' 'bar' ...></span>
<span class="gp">>>> </span><span class="n">InfSequence</span><span class="o">.</span><span class="n">fibonacci</span><span class="p">()</span>
<span class="go"><InfSequence: 0 1 1 2 3 5 ...></span>
</code></pre></div>
<p>Slicing works:</p>
<div class="highlight"><pre><span></span><code><span class="gp">>>> </span><span class="n">InfSequence</span><span class="p">(</span><span class="mi">5</span><span class="p">,</span> <span class="mi">6</span><span class="p">,</span> <span class="o">...</span><span class="p">)[</span><span class="mi">5</span><span class="p">:]</span>
<span class="go"><InfSequence: 10 11 12 13 14 15 ...></span>
<span class="gp">>>> </span><span class="n">InfSequence</span><span class="p">(</span><span class="mi">5</span><span class="p">,</span> <span class="mi">6</span><span class="p">,</span> <span class="o">...</span><span class="p">)[</span><span class="mi">5</span><span class="p">::</span><span class="mi">2</span><span class="p">]</span>
<span class="go"><InfSequence: 10 12 14 16 18 20 ...></span>
</code></pre></div>
<p>A somewhat reverse operation:</p>
<div class="highlight"><pre><span></span><code><span class="gp">>>> </span><span class="p">(</span><span class="mi">2</span><span class="p">,</span> <span class="mi">3</span><span class="p">,</span> <span class="mi">4</span><span class="p">)</span> <span class="o">+</span> <span class="n">InfSequence</span><span class="p">(</span><span class="mi">5</span><span class="p">,</span> <span class="mi">6</span><span class="p">,</span> <span class="o">...</span><span class="p">)</span>
<span class="go"><InfSequence: 2 3 4 5 6 7 ...></span>
</code></pre></div>
<p>Various arithmetic operations are supported:</p>
<div class="highlight"><pre><span></span><code><span class="gp">>>> </span><span class="n">InfSequence</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="o">...</span><span class="p">)</span> <span class="o">**</span> <span class="mi">2</span>
<span class="go"><InfSequence: 1 4 9 16 25 36 ...></span>
<span class="gp">>>> </span><span class="n">InfSequence</span><span class="p">(</span><span class="mi">3</span><span class="p">,</span> <span class="mi">5</span><span class="p">,</span> <span class="o">...</span><span class="p">)</span> <span class="o">-</span> <span class="n">InfSequence</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="o">...</span><span class="p">)</span>
<span class="go"><InfSequence: 2 3 4 5 6 7 ...></span>
<span class="gp">>>> </span><span class="n">InfSequence</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="o">...</span><span class="p">)</span><span class="o">.</span><span class="n">partial_sum</span><span class="p">(</span><span class="mi">10</span><span class="p">)</span>
<span class="go">55</span>
</code></pre></div>
<p>More magic methods:</p>
<div class="highlight"><pre><span></span><code><span class="gp">>>> </span><span class="n">InfSequence</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="o">...</span><span class="p">)</span><span class="o">.</span><span class="n">accumulate</span><span class="p">()</span>
<span class="go"><InfSequence: 1 3 6 10 15 21 ...></span>
<span class="gp">>>> </span><span class="n">InfSequence</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="o">...</span><span class="p">)</span> <span class="o">@</span> <span class="n">InfSequence</span><span class="p">(</span><span class="mi">1</span><span class="p">)</span>
<span class="go"><InfSequence: 1 4 9 16 25 36 ...></span>
</code></pre></div>
<p>For the complete API overview, please look at the <a href="https://pypi.org/project/infseq/">documentation</a>.</p>
<p>You can install the module from <a href="https://pypi.org/project/infseq/">PyPI</a>. It is licensed under 3-clause BSD
license. The source code is available on <a href="https://github.com/mitya57/infseq">GitHub</a>.</p>GNOME Flashback 3.16 available in archive, needs your help2015-07-22T00:00:00+03:002015-07-22T00:00:00+03:00Dmitry Shachnevtag:mitya57.me,2015-07-22:/weblog/2015/07/gnome-flashback-316-in-testing.html<p>Some time ago GNOME Flashback 3.16/3.17 packages landed in Debian testing
and Ubuntu wily.</p>
<p>GNOME Flashback is the project which continues the development of components
of classic GNOME session, including the GNOME Panel, the Metacity window
manager, and so on.</p>
<p><img alt="Screenshot" src="/static/flashback-screenshot.png" style="width: 100%;"></p>
<p>The full changelog can be found in …</p><p>Some time ago GNOME Flashback 3.16/3.17 packages landed in Debian testing
and Ubuntu wily.</p>
<p>GNOME Flashback is the project which continues the development of components
of classic GNOME session, including the GNOME Panel, the Metacity window
manager, and so on.</p>
<p><img alt="Screenshot" src="/static/flashback-screenshot.png" style="width: 100%;"></p>
<p>The full changelog can be found in <a href="https://mail.gnome.org/archives/gnome-flashback-list/2015-April/msg00004.html">official announcement mail by Alberts</a>
and in <code>changelog.gz</code> files in each package, but I want to list the most
imporant improvements in this release (compared to 3.14):</p>
<ul>
<li>
<p><a href="https://wiki.gnome.org/Projects/GnomePanel">GNOME Panel</a> and <a href="https://wiki.gnome.org/Projects/GnomeApplets">GNOME Applets</a> (uploaded version 3.16.1):</p>
<ul>
<li>
<p>The ability to use transparent panels has been restored.</p>
</li>
<li>
<p>The <a href="https://projects-old.gnome.org/netspeed/">netspeed applet</a> has been ported to the new API and integrated
into gnome-applets source tree.</p>
</li>
<li>
<p>Many deprecation warnings have been fixed, and the code has been
modernized.</p>
</li>
<li>
<p>This required a transition and a port of many third-party applets.
Currently in Debian these third-party applets are compatible with
gnome-panel 3.16: <a href="https://tracker.debian.org/pkg/command-runner-applet">command-runner-applet</a>, <a href="https://tracker.debian.org/pkg/gnubiff">gnubiff</a>,
<a href="https://tracker.debian.org/pkg/sensors-applet">sensors-applet</a>, <a href="https://tracker.debian.org/pkg/uim">uim</a>, <a href="https://tracker.debian.org/pkg/workrave">workrave</a>.</p>
</li>
</ul>
</li>
<li>
<p><a href="https://wiki.gnome.org/Projects/GnomeFlashback#FlashbackModule">GNOME Flashback</a> helper application (uploaded version 3.17.2):</p>
<ul>
<li>
<p>Added support for the on-screen display (OSD) when switching
brightness, volume, etc.</p>
</li>
<li>
<p>Applications using GNOME AppMenu are now shown correctly.</p>
</li>
</ul>
</li>
<li>
<p><a href="https://wiki.gnome.org/Projects/Metacity">Metacity</a> window manager (uploaded version 3.17.2):</p>
<ul>
<li>
<p>Metacity can now draw the window decorations based on the Gtk+ theme
(without need to add Metacity-specific theme). This follows Mutter
behavior, but (unlike Mutter) ability to use Metacity themes has been
preserved.</p>
</li>
<li>
<p>Adwaita and HighContrast themes for Metacity have been removed from
gnome-themes-standard, so they are now shipped as part of Metacity (in
<code>metacity-common</code> package).</p>
</li>
<li>
<p>Metacity now supports invisible window borders (the default setting is
10px extra space for resize cursor area).</p>
</li>
</ul>
</li>
</ul>
<p><strong>Sounds interesting? Contribute!</strong></p>
<p>If you are interested in helping us, please write to our mailing list:
<a href="mailto:gnome-flashback-list@gnome.org">gnome-flashback-list@gnome.org</a>.</p>
<p>The current TODO list is:</p>
<ol>
<li>Notification Daemon needs GTK notification support.</li>
<li>GNOME Flashback needs screenshot, screencast, keyboard layout switching and bluetooth status icon.</li>
<li>Fix/replace deprecated function usage in all modules.</li>
<li>libstatus-notifier — get it in usable state, create a new applet for gnome-panel.</li>
</ol>