Bridgetown2024-02-03T16:19:41+00:00/feed.xmlSimon NeutertI am a Ruby (on Rails) developer and open source enthusiast. Super interest in Clojure. Based in Ingelheim, next to Mainz and Wiesbaden (Rhein-Main).The sass gem is dead, long live: sass-embedded-host-ruby2024-01-13T00:00:00+00:002024-01-13T00:00:00+00:00repo://posts.collection/_posts/2024/2024-01-13-sass-embedded.md<p>I had this deprecation warning in my template sinatra project on GitHub for a while now.</p>
<p>The project should have ActiveRecord and Sass support, with sprockets and a few other things.</p>
<p>And man was this sass topic a rabbit hole in one way or another. Because the hint in the deprecation warning didn’t work out of the box.</p>
<p>And with most people being on Rails, or when on Sinatra not very interested in Sass, or they know how to fix it, I had mostly trial and error to go on.</p>
<p>Here is how I setup my project’s/app’s environment from day one:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">class</span> <span class="nc">MyApp</span> <span class="o"><</span> <span class="no">Sinatra</span><span class="o">::</span><span class="no">Base</span>
<span class="c1"># initialize new sprockets environment</span>
<span class="n">set</span> <span class="ss">:environment</span><span class="p">,</span> <span class="no">Sprockets</span><span class="o">::</span><span class="no">Environment</span><span class="p">.</span><span class="nf">new</span>
<span class="c1"># append assets paths</span>
<span class="n">environment</span><span class="p">.</span><span class="nf">append_path</span> <span class="s1">'assets/stylesheets'</span>
<span class="n">environment</span><span class="p">.</span><span class="nf">append_path</span> <span class="s1">'assets/javascripts'</span>
<span class="c1"># compress assets</span>
<span class="n">environment</span><span class="p">.</span><span class="nf">js_compressor</span> <span class="o">=</span> <span class="ss">:uglify</span>
<span class="n">environment</span><span class="p">.</span><span class="nf">css_compressor</span> <span class="o">=</span> <span class="ss">:scss</span> <span class="c1"># << here's the problem</span>
<span class="c1"># get assets</span>
<span class="n">get</span> <span class="s1">'/assets/*'</span> <span class="k">do</span>
<span class="n">env</span><span class="p">[</span><span class="s1">'PATH_INFO'</span><span class="p">].</span><span class="nf">sub!</span><span class="p">(</span><span class="s1">'/assets'</span><span class="p">,</span> <span class="s1">''</span><span class="p">)</span>
<span class="n">settings</span><span class="p">.</span><span class="nf">environment</span><span class="p">.</span><span class="nf">call</span><span class="p">(</span><span class="n">env</span><span class="p">)</span>
<span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>
<p>I highlighted the problem line. It’s the <code class="highlighter-rouge">:scss</code> compressor. This simply doesn’t work anymore. And all the deprecation warning says is:</p>
<p><code class="highlighter-rouge">switch to the sassc ruby gem</code></p>
<p>well, I did that, and it is part of the solution, but the glue was missing.</p>
<p><code class="highlighter-rouge">$ bundle add sass-embedded sassc</code></p>
<p>https://github.com/sass-contrib/sass-embedded-host-ruby</p>
<p>Then require both gems, remove <code class="highlighter-rouge">sass</code> from the Gemfile, and remove the <code class="highlighter-rouge">:scss</code> compressor line.</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">class</span> <span class="nc">MyApp</span> <span class="o"><</span> <span class="no">Sinatra</span><span class="o">::</span><span class="no">Base</span>
<span class="c1"># initialize new sprockets environment</span>
<span class="n">set</span> <span class="ss">:environment</span><span class="p">,</span> <span class="no">Sprockets</span><span class="o">::</span><span class="no">Environment</span><span class="p">.</span><span class="nf">new</span>
<span class="c1"># append assets paths</span>
<span class="n">environment</span><span class="p">.</span><span class="nf">append_path</span> <span class="s1">'assets/stylesheets'</span>
<span class="n">environment</span><span class="p">.</span><span class="nf">append_path</span> <span class="s1">'assets/javascripts'</span>
<span class="c1"># compress assets</span>
<span class="n">environment</span><span class="p">.</span><span class="nf">js_compressor</span> <span class="o">=</span> <span class="ss">:uglify</span>
<span class="c1"># 👇 remove any css compressor settings you might stumble upon, be it sinatra, roda or rails</span>
<span class="c1"># environment.css_compressor = :scss # << here's the problem</span>
<span class="c1"># get assets</span>
<span class="n">get</span> <span class="s1">'/assets/*'</span> <span class="k">do</span>
<span class="n">env</span><span class="p">[</span><span class="s1">'PATH_INFO'</span><span class="p">].</span><span class="nf">sub!</span><span class="p">(</span><span class="s1">'/assets'</span><span class="p">,</span> <span class="s1">''</span><span class="p">)</span>
<span class="n">settings</span><span class="p">.</span><span class="nf">environment</span><span class="p">.</span><span class="nf">call</span><span class="p">(</span><span class="n">env</span><span class="p">)</span>
<span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>
<p>Checkout the pull request for the full diff:<br />
https://github.com/simonneutert/sinatras-skeleton/pull/84/files</p>
<p>And that’s it. I hope this helps someone else out there.</p>Setup FiraCode Nerd Font Mono with ligatures in VSCode for MacOS2024-01-07T00:00:00+00:002024-01-07T00:00:00+00:00repo://posts.collection/_posts/2024/2024-01-07-vscode-fira.md<p>In <code class="highlighter-rouge">settings.json</code> have the following values:</p>
<div class="language-json highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="p">{</span><span class="w">
</span><span class="err">//</span><span class="w"> </span><span class="err">shortened</span><span class="w"> </span><span class="err">for</span><span class="w"> </span><span class="err">brevity</span><span class="w">
</span><span class="nl">"editor.fontFamily"</span><span class="p">:</span><span class="w"> </span><span class="s2">"'FiraCode Nerd Font Mono'"</span><span class="p">,</span><span class="w">
</span><span class="nl">"editor.fontSize"</span><span class="p">:</span><span class="w"> </span><span class="mi">13</span><span class="p">,</span><span class="w">
</span><span class="nl">"editor.fontLigatures"</span><span class="p">:</span><span class="w"> </span><span class="kc">true</span><span class="w">
</span><span class="p">}</span><span class="w">
</span></code></pre></div></div>
<p>I used the builtin Font Manager to install my nerdy fonts beforehand.</p>
<p>Get them here: https://github.com/ryanoasis/nerd-fonts</p>Elevate Your Podcast Experience: Using Apple Shortcuts to Play Random Episodes (and NGINX)2023-12-18T00:00:00+00:002023-12-18T00:00:00+00:00repo://posts.collection/_posts/2023/2023-12-18-random-podcast-episodes.md<p>Podcasts have become an integral part of my daily routine, and accessing them seamlessly is crucial for a smooth listening experience. In this guide, I’ll explore how you can leverage Nginx to offer a JSON file containing links to podcast episodes.</p>
<p>Additionally, I’ll demonstrate how iPhone users can use the Shortcuts app to load the JSON file, extract a random entry, and open the URL for their preferred podcast service.</p>
<h2 id="the-following-is-one-of-many-ways--tldr">The following is one of many ways / tldr</h2>
<p>It’s up to you as a user or the podcaster to host your episodes in a way that allows you to access them via a JSON file. Then load this via the <strong>Shortcut</strong> app and use the <strong>Get Dictionary Value</strong> action to extract a random entry from the JSON file. Finally, use the <strong>Open URLs</strong> action to open the URL in your preferred podcast app.</p>
<p><strong>The following code will be pseudo code and just guide you a little.</strong></p>
<h3 id="setting-up-nginx-to-serve-json">Setting Up Nginx to Serve JSON</h3>
<p>Nginx is a powerful web server that can be used to serve static files, including JSON. Here’s a quick example of an Nginx configuration to serve a podcast.json file:</p>
<div class="language-nginx highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">server</span> <span class="p">{</span>
<span class="kn">listen</span> <span class="mi">80</span><span class="p">;</span>
<span class="kn">server_name</span> <span class="s">yourdomain.com</span><span class="p">;</span>
<span class="kn">location</span> <span class="n">/podcasts</span> <span class="p">{</span>
<span class="kn">alias</span> <span class="n">/path/to/your/json/files</span><span class="p">;</span>
<span class="kn">index</span> <span class="s">podcast.json</span><span class="p">;</span>
<span class="p">}</span>
<span class="p">}</span>
</code></pre></div></div>
<p>Replace yourdomain.com with your actual domain and adjust the path accordingly. Make sure the JSON file contains an array of podcast entries, each with a title and url pointing to the podcast episode.</p>
<h3 id="creating-the-podcast-json-file">Creating the Podcast JSON File</h3>
<p>Here’s an example of how your podcast.json file might look:</p>
<div class="language-json highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="p">[</span><span class="w">
</span><span class="p">{</span><span class="w">
</span><span class="nl">"title"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Podcast Episode 1"</span><span class="p">,</span><span class="w">
</span><span class="nl">"url"</span><span class="p">:</span><span class="w"> </span><span class="s2">"https://spotify.com/episode1"</span><span class="w">
</span><span class="p">},</span><span class="w">
</span><span class="p">{</span><span class="w">
</span><span class="nl">"title"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Podcast Episode 2"</span><span class="p">,</span><span class="w">
</span><span class="nl">"url"</span><span class="p">:</span><span class="w"> </span><span class="s2">"https://applepodcasts.com/episode2"</span><span class="w">
</span><span class="p">},</span><span class="w">
</span><span class="err">//</span><span class="w"> </span><span class="err">Add</span><span class="w"> </span><span class="err">more</span><span class="w"> </span><span class="err">entries</span><span class="w"> </span><span class="err">as</span><span class="w"> </span><span class="err">needed</span><span class="w">
</span><span class="p">]</span><span class="w">
</span></code></pre></div></div>
<p>An Array of objects.</p>
<h4 id="using-shortcuts-on-iphone">Using Shortcuts on iPhone</h4>
<ol>
<li>Get the <a href="https://apps.apple.com/us/app/shortcuts/id915249334">Shortcuts app</a> from the App Store.</li>
<li>Setup a new shortcut</li>
</ol>
<p>Then, add the following actions to your shortcut:</p>
<ol>
<li>Get contents of <code class="highlighter-rouge">https://yourdomain.com/podcasts/podcast.json</code></li>
<li>Get <code class="highlighter-rouge">Random Item</code> from <code class="highlighter-rouge">Get Contents of URL</code></li>
<li>Get <code class="highlighter-rouge">Dictionary</code> from <code class="highlighter-rouge">Item from List</code></li>
<li>Get <code class="highlighter-rouge">Random Item</code> from <code class="highlighter-rouge">Dictionary</code></li>
<li>Get <code class="highlighter-rouge">Value</code> for <code class="highlighter-rouge">your-key-in-the-json</code> from <code class="highlighter-rouge">Dictionary</code></li>
<li>Open <code class="highlighter-rouge">Dictionary Value</code></li>
</ol>
<hr />
<style>
#shuffle-gag-screenshot {
max-width: 90%;
height: auto;
max-height: 480px;
}
</style>
<p><img src="/images/2023/shuffle-gag.jpeg" id="shuffle-gag-screenshot" alt="Shortcuts App showing an example workflow to load and extract a sample podcast from an array of dictionaries" /></p>
<p>Tada 🥳🪄✨</p>I coded a cash register backend in Ruby backed by Postgres2023-06-26T00:00:00+00:002023-06-26T00:00:00+00:00repo://posts.collection/_posts/2023/2023-06-26-cash-ka-ching.md<p>I’ve been working on a cash register backend in Ruby for the past few months after work and sometimes on weekends. It’s been a fun project and I’ve learned a lot about Ruby, Postgres, and how to keep calm and trash the hell out of ideas.</p>
<p>Again, as readers of my blog will guess, I bet my horses on Roda and Sequel. I’ve been using them for a while now and I’m very happy with them. I’ve also been using Postgres for a while now and I’m very happy with it too. I’m not sure if I’ll ever use MySQL again 🤭</p>
<p>My main idea was to play with Postgres’ locking mechanisms and see if I could build a cash register backend that could handle multiple concurrent requests. I wanted to eliminate the ever so slight chance of concurrent requests messing up the cash register’s state. I also wanted to see if this would slim down the logic part in the backend code.</p>
<h4 id="tldr">TLDR</h4>
<p>I think I’ve succeeded in building a cash register backend that can handle multiple concurrent requests. I’ve also managed to slim down the logic part in the backend code. I’m not sure if someone will ever use this in production, but it’s been a fun project and I’ve learned a lot.</p>
<ul>
<li><a href="https://github.com/simonneutert/ka-ching-backend">ka-ching-backend</a></li>
<li><a href="https://github.com/simonneutert/ka-ching-client">ka-ching-client</a></li>
</ul>
<p>There is also a demo to showcase the API and the client in conjunction. It’s a simple Roda app with <a href="https://htmx.org/">htmx</a> for the frontend. It’s not pretty, but it works.</p>
<p><a href="https://github.com/simonneutert/ka-ching-demo">Check out the demo repository on GitHub</a>!</p>
<h2 id="table-of-contents">Table of contents<!-- omit in toc --></h2>
<ul>
<li><a href="#the-cash-register">The cash register</a>
<ul>
<li><a href="#what-this-this-system-is-not-intended-to-bedo">What this this system is not intended to be/do</a></li>
</ul>
</li>
<li><a href="#its-architecture-a-micro-monolith">It’s Architecture? A Micro-Monolith!</a></li>
<li><a href="#implementing-the-backend-with-roda-and-sequel">Implementing the backend with Roda and Sequel</a>
<ul>
<li><a href="#my-goals-for-the-backend">My goals for the backend</a></li>
<li><a href="#database-lockings-with-sequel">Database Lockings with Sequel</a></li>
<li><a href="#learning-more-about-roda-and-sequel">Learning more about Roda and Sequel</a></li>
</ul>
</li>
</ul>
<h2 id="the-cash-register">The cash register</h2>
<p>The cash register is a simple machine that can handle the following operations:</p>
<ul>
<li>Wrap transactions of the cash register for a certain period of time</li>
<li>Add money to the cash register (deposit)</li>
<li>Remove money from the cash register (withdraw)</li>
<li>Get the current state of the cash register (saldo)</li>
<li>Get the history of the cash register (list of transactions)</li>
<li>Track certain events in an audit log (e.g. when the cash register was reopened, after it was closed)</li>
</ul>
<h3 id="what-this-this-system-is-not-intended-to-bedo">What this this system is not intended to be/do</h3>
<ul>
<li>The cash register is not a point of sale (POS) system.</li>
<li>It’s not meant to be used by a cashier to scan items and print receipts.</li>
<li>It’s not meant to be used by a customer to pay for their groceries.</li>
<li>It’s not meant to be used by a manager to get reports on the sales of the day.</li>
</ul>
<h2 id="its-architecture-a-micro-monolith">It’s Architecture? A Micro-Monolith!</h2>
<p>I’ve been using the term “micro-monolith” for a while now. I’m not sure if it’s a thing, but I like it. It’s a monolith, but it’s a small one. It’s a monolith that can easily serve as a base of something bigger oder act as a standalone system. Being a micro-monolith it cannot be split into smaller parts. It’s a monolith, but it’s a small one.</p>
<p>The main reason I struggle with calling it a microservice is, that I want it to be understood as a product and not as a service. It does not do a generic single thing. It does a specific set of things that together represent my interpretation of a cash register system. Maybe I am just too happy with <a href="https://world.hey.com/dhh/we-stand-to-save-7m-over-five-years-from-our-cloud-exit-53996caa">DHH leaving the cloud</a> and <a href="https://www.reddit.com/r/aws/comments/137lyno/amazon_prime_microservices_to_monolith/">Amazon reverting to a monolith</a> for their “Prime Video”. I don’t know.</p>
<h2 id="implementing-the-backend-with-roda-and-sequel">Implementing the backend with Roda and Sequel</h2>
<p>It’s been a breeze! By coincidence Jeremy Evans, the maintainer of Roda and Sequel (and many more) had been the guest in <a href="https://www.youtube.com/watch?v=Hh_lqARFGcg">The Rubber Duck Dev Show</a> sharing his ideas first hand with the audience.</p>
<h3 id="my-goals-for-the-backend">My goals for the backend</h3>
<ul>
<li>multi-tenant database architecture</li>
<li>relies on database locks for critical operations</li>
<li>json columns in most tables, to pass your own context</li>
<li>fast, cheap, and reliable</li>
<li>short and concise codebase</li>
<li>lean on dependencies</li>
<li>with containerization in mind</li>
<li>have a client ready to use the API as a ruby gem</li>
</ul>
<h3 id="database-lockings-with-sequel">Database Lockings with Sequel</h3>
<p>In order to lock a table in Postgres you can use the <code class="highlighter-rouge">LOCK</code> statement. It’s a very powerful tool and can be used to lock a table in different ways. I’ve been using the <code class="highlighter-rouge">ACCESS EXCLUSIVE</code> mode to lock the tables <code class="highlighter-rouge">lockings</code> and <code class="highlighter-rouge">bookings</code> in order to prevent concurrent requests from messing up the cash register’s state.</p>
<p>Looking at the code below you can see that I’m using Sequel’s <code class="highlighter-rouge">transaction</code> method to wrap the two <code class="highlighter-rouge">LOCK</code> statements and the <code class="highlighter-rouge">INSERT</code> statement into a single transaction. This way I can be sure that the two <code class="highlighter-rouge">LOCK</code> statements and the <code class="highlighter-rouge">INSERT</code> statement are executed in a single transaction. If one of the statements fails, the whole transaction is rolled back.</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="vi">@conn</span><span class="p">.</span><span class="nf">transaction</span> <span class="k">do</span>
<span class="vi">@conn</span><span class="p">.</span><span class="nf">run</span><span class="p">(</span><span class="s1">'LOCK TABLE lockings IN ACCESS EXCLUSIVE MODE'</span><span class="p">)</span>
<span class="vi">@conn</span><span class="p">.</span><span class="nf">run</span><span class="p">(</span><span class="s1">'LOCK TABLE bookings IN ACCESS EXCLUSIVE MODE'</span><span class="p">)</span>
<span class="n">new_booking_id</span> <span class="o">=</span> <span class="vi">@conn_bookings</span><span class="p">.</span><span class="nf">insert</span><span class="p">(</span><span class="ss">id: </span><span class="no">SecureRandom</span><span class="p">.</span><span class="nf">uuid</span><span class="p">,</span>
<span class="ss">amount_cents: </span><span class="vi">@booker</span><span class="p">.</span><span class="nf">amount_cents</span><span class="p">,</span>
<span class="ss">action: </span><span class="vi">@booker</span><span class="p">.</span><span class="nf">action</span><span class="p">,</span>
<span class="ss">realized: </span><span class="vi">@booker</span><span class="p">.</span><span class="nf">realized</span><span class="p">,</span>
<span class="ss">context: </span><span class="vi">@booker</span><span class="p">.</span><span class="nf">context</span><span class="p">.</span><span class="nf">to_json</span><span class="p">)</span>
<span class="n">query_bookings</span><span class="p">(</span><span class="vi">@conn</span><span class="p">).</span><span class="nf">find_by</span><span class="p">(</span><span class="ss">id: </span><span class="n">new_booking_id</span><span class="p">)</span>
<span class="k">end</span>
</code></pre></div></div>
<p>If you want to dig into what locking a table in Postgres means, I recommend reading <a href="https://www.postgresql.org/docs/current/explicit-locking.html">the Postgres Documentation on Explicit Locking</a>.</p>
<h3 id="learning-more-about-roda-and-sequel">Learning more about Roda and Sequel</h3>
<p>As the two projects play well together I learned a lot about both of them. Roda’s idea of the routing tree was a great opportunity to come up with a way of using and reusing database connections depending on where your requests are routed to. Sequel keeps things easy and straight by mostly acting as a wrapper around SQL, without applying custom object-relational mapping (ORM) logic. Hashes are the way to go and I really dig that.</p>
<p>Connecting to one of the tenant database’s looks like this currenyly:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">def</span> <span class="nc">self</span><span class="o">.</span><span class="nf">db_connection</span><span class="p">(</span><span class="n">database</span><span class="p">,</span> <span class="o">&</span><span class="n">block</span><span class="p">)</span>
<span class="no">Sequel</span><span class="p">.</span><span class="nf">connect</span><span class="p">(</span>
<span class="s2">"postgres://</span><span class="si">#{</span><span class="no">DATABASE_URL</span><span class="si">}</span><span class="s2">:</span><span class="si">#{</span><span class="no">DATABASE_PORT</span><span class="si">}</span><span class="s2">/</span><span class="si">#{</span><span class="n">database</span><span class="si">}</span><span class="s2">?user=</span><span class="si">#{</span><span class="no">DATABASE_USER</span><span class="si">}</span><span class="s2">&password=</span><span class="si">#{</span><span class="no">DATABASE_PASSWORD</span><span class="si">}</span><span class="s2">"</span><span class="p">,</span>
<span class="ss">logger: </span><span class="no">DB</span><span class="o">::</span><span class="no">LOGGER</span><span class="p">,</span>
<span class="o">&</span><span class="n">block</span>
<span class="p">)</span>
<span class="k">end</span>
</code></pre></div></div>
<p>Utilizing the <code class="highlighter-rouge">&block</code> parameter of the <code class="highlighter-rouge">Sequel.connect</code> method allows me to pass a block to the method. This way I can make sure that the connection is closed after the block has been executed. This is a great way to make sure that the connection is closed after the request has been processed.</p>use minimagick convert pdf frontpage to png2023-06-25T00:00:00+00:002023-06-25T00:00:00+00:00repo://posts.collection/_posts/2023/2023-06-25-pdf-png-preview.md<p>Use your majestic brain waves to figure out how where to get the pdf content from. Then refactor the code to your needs. This is just a snippet.</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">def</span> <span class="nf">to_pdf_preview_png_base64</span>
<span class="k">begin</span>
<span class="n">base64string</span> <span class="o">=</span> <span class="s2">""</span>
<span class="no">Tempfile</span><span class="p">.</span><span class="nf">create</span><span class="p">([</span><span class="s2">"filename_randomized"</span><span class="p">,</span> <span class="s1">'.png'</span><span class="p">],</span> <span class="ss">binmode: </span><span class="kp">true</span><span class="p">)</span> <span class="k">do</span> <span class="o">|</span><span class="n">temp_png_file</span><span class="o">|</span>
<span class="no">Tempfile</span><span class="p">.</span><span class="nf">create</span><span class="p">([</span><span class="s2">"filename_randomized"</span><span class="p">,</span> <span class="s1">'.pdf'</span><span class="p">],</span> <span class="ss">binmode: </span><span class="kp">true</span><span class="p">)</span> <span class="k">do</span> <span class="o">|</span><span class="n">temp_pdf_file</span><span class="o">|</span>
<span class="n">temp_pdf_file</span><span class="p">.</span><span class="nf">write</span><span class="p">(</span><span class="n">pdf_content</span><span class="p">)</span> <span class="c1"># << you know where to get this from</span>
<span class="n">pdf</span> <span class="o">=</span> <span class="no">MiniMagick</span><span class="o">::</span><span class="no">Image</span><span class="p">.</span><span class="nf">open</span><span class="p">(</span><span class="n">temp_pdf_file</span><span class="p">.</span><span class="nf">path</span><span class="p">)</span>
<span class="n">png_file</span> <span class="o">=</span> <span class="no">MiniMagick</span><span class="o">::</span><span class="no">Tool</span><span class="o">::</span><span class="no">Convert</span><span class="p">.</span><span class="nf">new</span> <span class="k">do</span> <span class="o">|</span><span class="n">convert</span><span class="o">|</span>
<span class="n">convert</span><span class="p">.</span><span class="nf">background</span> <span class="s2">"white"</span>
<span class="n">convert</span><span class="p">.</span><span class="nf">flatten</span>
<span class="n">convert</span><span class="p">.</span><span class="nf">density</span> <span class="mi">150</span>
<span class="n">convert</span><span class="p">.</span><span class="nf">quality</span> <span class="mi">100</span>
<span class="n">convert</span><span class="p">.</span><span class="nf">format</span> <span class="s2">"png"</span>
<span class="n">convert</span> <span class="o"><<</span> <span class="n">pdf</span><span class="p">.</span><span class="nf">pages</span><span class="p">.</span><span class="nf">first</span><span class="p">.</span><span class="nf">path</span>
<span class="n">convert</span> <span class="o"><<</span> <span class="n">temp_png_file</span><span class="p">.</span><span class="nf">path</span>
<span class="k">end</span>
<span class="n">png_contents</span> <span class="o">=</span> <span class="no">File</span><span class="p">.</span><span class="nf">binread</span><span class="p">(</span><span class="n">temp_png_file</span><span class="p">.</span><span class="nf">path</span><span class="p">)</span>
<span class="n">base64string</span> <span class="o">=</span> <span class="s2">"data:image/png;base64,</span><span class="si">#{</span><span class="no">Base64</span><span class="p">.</span><span class="nf">encode64</span><span class="p">(</span><span class="n">png_contents</span><span class="p">)</span><span class="si">}</span><span class="s2">"</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="n">base64string</span>
<span class="k">rescue</span> <span class="o">=></span> <span class="n">e</span>
<span class="nb">puts</span> <span class="n">e</span>
<span class="c1"># empty pixel https://png-pixel.com/</span>
<span class="s2">"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAADUlEQVR42mP8/5+hHgAHggJ/PchI7wAAAABJRU5ErkJggg== "</span>
<span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>Ruby REST API client for Papierkram.de2023-04-12T00:00:00+00:002023-04-12T00:00:00+00:00repo://posts.collection/_posts/2023/2023-04-12-pkapi.md<p>I have written a small REST API client for the Papierkram.de API, which is a simple wrapper around the Faraday gem. Although it’s not perfect, it works well. For the HTTP client library, I chose <a href="https://github.com/HoneyryderChuck/httpx">httpx</a>, which promises to be the future with blazing-fast performance, and it’s free too!</p>
<p>tldr; <a href="https://rubygems.org/gems/papierkram_api_client">here’s the gem</a> and <a href="https://github.com/simonneutert/papierkram_api_client">here’s the code</a>.</p>
<p>In this post, I would like to share some of the things I have learned while writing this gem.</p>
<p>Yet, there’s still some more documentation and testing to do. 😅</p>
<h2 id="testing-with-vcr-and-minitest">Testing with VCR and Minitest</h2>
<p>I chose to use Minitest for testing, as I am a big fan of its simplicity and think it is a great choice for testing. One of the things I like about it is that it is just plain Ruby, with no DSL, magic, or weird syntax. While I’m not certain whether it is actually faster than RSpec, it certainly feels that way.</p>
<p>Minitest pairs very well with <a href="https://github.com/vcr/vcr">VCR</a> for testing the API. Personally, I am not a big fan of mocking, so I prefer to record the API responses and replay them during testing. However, it is important to keep an eye on your recorded data to ensure that it is still valid.</p>
<p>Setting up Minitest and VCR is straightforward using <a href="https://github.com/mfpiccolo/minitest-vcr">minitest-vcr</a>.</p>
<p>Here’s an example setup of Minitest’s <code class="highlighter-rouge">test_helper.rb</code>:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1"># $ cat app_name/test/test_helper.rb</span>
<span class="nb">require</span> <span class="s2">"minitest/autorun"</span>
<span class="nb">require</span> <span class="s2">"minispec-metadata"</span>
<span class="nb">require</span> <span class="s2">"vcr"</span>
<span class="nb">require</span> <span class="s2">"minitest-vcr"</span>
<span class="nb">require</span> <span class="s2">"faraday"</span>
<span class="no">VCR</span><span class="p">.</span><span class="nf">configure</span> <span class="k">do</span> <span class="o">|</span><span class="n">c</span><span class="o">|</span>
<span class="n">c</span><span class="p">.</span><span class="nf">cassette_library_dir</span> <span class="o">=</span> <span class="s1">'test/cassettes'</span>
<span class="n">c</span><span class="p">.</span><span class="nf">hook_into</span> <span class="ss">:faraday</span>
<span class="k">end</span>
<span class="no">MinitestVcr</span><span class="o">::</span><span class="no">Spec</span><span class="p">.</span><span class="nf">configure!</span>
</code></pre></div></div>
<p>It will record the request and the response. But it will also record the request headers. And if you don’t pay attention,
you might end up with a recorded response that leaks personal or credential data.</p>
<p>So make sure to filter out any sensitive data from the request headers.</p>
<p>VCR brings some features that will help <a href="https://benoittgt.github.io/vcr/#/configuration/filter_sensitive_data">you with that</a>.
You can filter out parts of the request headers, like this:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">interaction</span><span class="p">.</span><span class="nf">response</span><span class="p">.</span><span class="nf">headers</span><span class="p">[</span><span class="s1">'set-cookie'</span><span class="p">].</span><span class="nf">first</span><span class="p">.</span><span class="nf">split</span><span class="p">(</span><span class="s1">';'</span><span class="p">).</span><span class="nf">first</span><span class="p">.</span><span class="nf">split</span><span class="p">(</span><span class="s1">'='</span><span class="p">).</span><span class="nf">last</span>
</code></pre></div></div>
<p>OR you can filter out the whole request headers:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="no">VCR</span><span class="p">.</span><span class="nf">configure</span> <span class="k">do</span> <span class="o">|</span><span class="n">c</span><span class="o">|</span>
<span class="c1"># ...</span>
<span class="n">c</span><span class="p">.</span><span class="nf">filter_sensitive_data</span><span class="p">(</span><span class="s1">'<API_KEY>'</span><span class="p">)</span> <span class="k">do</span> <span class="o">|</span><span class="n">interaction</span><span class="o">|</span>
<span class="n">interaction</span><span class="p">.</span><span class="nf">request</span><span class="p">.</span><span class="nf">headers</span><span class="p">[</span><span class="s1">'X-Http-Username'</span><span class="p">]</span>
<span class="k">end</span>
<span class="c1"># ...</span>
<span class="k">end</span>
</code></pre></div></div>
<p>Filter out a secret value coming from the environment:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="no">VCR</span><span class="p">.</span><span class="nf">configure</span> <span class="k">do</span> <span class="o">|</span><span class="n">c</span><span class="o">|</span>
<span class="c1"># ...</span>
<span class="n">c</span><span class="p">.</span><span class="nf">filter_sensitive_data</span><span class="p">(</span><span class="s1">'<API_KEY>'</span><span class="p">)</span> <span class="p">{</span> <span class="no">ENV</span><span class="p">[</span><span class="s1">'PAPIERKRAM_API_KEY'</span><span class="p">]</span> <span class="p">}</span>
<span class="c1"># ...</span>
<span class="k">end</span>
</code></pre></div></div>
<p>This will help you avoid leaking sensitive data that you are aware of and can pin down exactly.</p>
<p>Filtering emails is a bit more tricky. You can filter out the whole email address in most cases something like this:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1"># tests ...</span>
<span class="no">VCR</span><span class="p">.</span><span class="nf">configure</span> <span class="k">do</span> <span class="o">|</span><span class="n">c</span><span class="o">|</span>
<span class="c1"># ...</span>
<span class="n">c</span><span class="p">.</span><span class="nf">before_record</span> <span class="k">do</span> <span class="o">|</span><span class="n">interaction</span><span class="o">|</span>
<span class="n">interaction</span><span class="p">.</span><span class="nf">response</span><span class="p">.</span><span class="nf">body</span><span class="p">.</span><span class="nf">scan</span><span class="p">(</span><span class="sr">/\b[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Za-z]{2,4}\b/</span><span class="p">).</span><span class="nf">each</span> <span class="k">do</span> <span class="o">|</span><span class="n">email</span><span class="o">|</span>
<span class="n">interaction</span><span class="p">.</span><span class="nf">response</span><span class="p">.</span><span class="nf">body</span><span class="p">.</span><span class="nf">gsub!</span><span class="p">(</span><span class="n">email</span><span class="p">,</span> <span class="s1">'<EMAIL>'</span><span class="p">)</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>
<h2 id="clean-your-cassette-files-once-in-a-while">Clean your cassette files once in a while</h2>
<p>I know there are auto-clean options in VCR, but in most cases, I can manage it myself. APIs usually don’t change that often or change gradually enough.</p>
<p>Deleting the cassettes a few times a year or after API changes have been communicated should be sufficient.</p>
<h2 id="check-out-the-code-of-the-gem">Check out the code of the gem</h2>
<p>If you want to see how I implemented the gem, you can check out the code on <a href="https://github.com/simonneutert/papierkram_api_client">GitHub</a>.
Please, provide feedback if you have any. I am always happy to learn something new. And if you have any questions, feel free to ask.</p>
<h2 id="conclusion">Conclusion</h2>
<p>It’s always fun to put something together that you can use in your own projects. Working on this gem was a great learning experience for me.
Implementing Minitest, VCR, configuring Faraday and Rubocop, direnv and setting up GitHub Actions was a great way to learn more about Ruby and its ecosystem.
I want to maintain a Changelog.md file for this gem, so I (you) can keep track of the changes and document them.
I will also add some more tests and documentation in the future.</p>Ruby: Sequel and Dates BC (Before Christ) - Cheesus Chwist, I nearly went mad 🤪2023-04-08T00:00:00+00:002023-04-08T00:00:00+00:00repo://posts.collection/_posts/2023/2023-04-08-before-christ-postgres-sequel.md<p>When fiddling with date ranges, I stumbled about an error which caused my application to break.</p>
<div class="language-text highlighter-rouge"><div class="highlight"><pre class="highlight"><code>PG::DatetimeFieldOverflow: ERROR: date/time field value out of range: "0000-12-31 23:00:00.000000+0000"
LINE 1: ..." WHERE (("action" = 'deposit') AND ("realized" > '0000-12-3...
^: SELECT sum(cast(amount_cents as int)) as saldo FROM "bookings" WHERE (("action" = 'deposit') AND ("realized" > '0000-12-31 23:00:00.000000+0000')) LIMIT 1
</code></pre></div></div>
<p>I configured Sequel (and the database) to use UTC all the way down, causing my testsuite to brake 🥲<br />
https://sequel.jeremyevans.net/rdoc/classes/Sequel/Timezones.html</p>
<h2 id="enhancing-postgresql-date-support-with-pg_extended_date_support-sequel-extension">Enhancing PostgreSQL Date Support with <code class="highlighter-rouge">pg_extended_date_support</code> Sequel extension</h2>
<p>PostgreSQL is a powerful and versatile open-source relational database management system. Its robust feature set has made it a popular choice for developers and businesses worldwide. One area where PostgreSQL has some limitations is in handling dates and timestamps, specifically infinite and BC dates/timestamps. I</p>
<p>What is Sequel’s <code class="highlighter-rouge">pg_extended_date_support</code> extension?</p>
<p>The <code class="highlighter-rouge">pg_extended_date_support</code> extension adds support for infinite and BC dates/timestamps in PostgreSQL. While the postgres adapter already had a convert_infinite_timestamps setting, it wasn’t supported in the jdbc/postgresql adapter, and it didn’t handle BC dates/timestamps. This new extension resolves these issues and ensures better compatibility with various adapters.</p>
<p>How to Use the <code class="highlighter-rouge">pg_extended_date_support</code> Extension:</p>
<p>To use the pg_extended_date_support extension, you will need to load it into your application and configure it according to your needs. By default, the extension only fixes the handling of BC dates/timestamps. To enable it to handle infinite timestamps, you need to choose the appropriate setting for your application. Here’s how to do it:</p>
<ol>
<li>Load the extension for your database:</li>
</ol>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="no">DB</span><span class="p">.</span><span class="nf">extension</span> <span class="ss">:pg_extended_date_support</span>
</code></pre></div></div>
<ol>
<li>Configure the convert_infinite_timestamps setting:</li>
</ol>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="no">DB</span><span class="p">.</span><span class="nf">convert_infinite_timestamps</span> <span class="o">=</span> <span class="ss">:string</span> <span class="c1"># or :float or :nil</span>
</code></pre></div></div>
<p>Choose the appropriate setting based on your application’s requirements. The available options are:</p>
<p>:string - Infinite timestamps will be converted to strings.
:float - Infinite timestamps will be converted to floating-point numbers.
:nil - Infinite timestamps will not be converted and will be returned as nil.
Additional Benefits of the pg_extended_date_support Extension:</p>
<h3 id="bonus--conclusion">Bonus / Conclusion</h3>
<p>Apart from handling infinite and BC dates/timestamps, the pg_extended_date_support extension also enables the handling of timezone offsets with seconds. This feature is not natively supported by Ruby’s Time class in versions earlier than 2.5. By using this extension, you can now work with timezones that have a non-minute offset, making your application more flexible and adaptable to various timezones.</p>
<p>The pg_extended_date_support extension is a valuable addition to PostgreSQL, as it improves the handling of dates and timestamps, particularly infinite and BC dates/timestamps. Additionally, it enables support for timezone offsets with seconds, further enhancing PostgreSQL’s capabilities. If you’re working with PostgreSQL, consider integrating this extension into your application to make the most of its robust date and timestamp handling features.</p>RuboCop ships with a server2023-03-08T00:00:00+00:002023-03-08T00:00:00+00:00repo://posts.collection/_posts/2023/2023-03-08-rubocop.md<p>For everybody running the standard MRI/C Ruby implementation, did you know that you can speed up your Rubocop checks by quite a bit?</p>
<blockquote>
<p>You can reduce the RuboCop boot time significantly (something like 850x faster) by using the –server command-line option.</p>
<p>The –server option speeds up the launch of the rubocop command by utilizing a standalone server process that loads the RuboCop runtime production files (i.e. require ‘rubocop’).</p>
<p>Normally RuboCop starts somewhat slowly because it needs to require a ton of files and that’s fairly slow. With the RuboCop server we sidestep this nasty issue and make it much more pleasant to interact with RuboCop from text editors and IDEs.</p>
<p>Source: https://docs.rubocop.org/rubocop/usage/server.html</p>
</blockquote>
<p>I recommend that you read the documentation. Please do not make the same mistake I did!</p>
<p>Make sure to add a <code class="highlighter-rouge">.rubocop</code> file to your project, but this time <strong>without the <code class="highlighter-rouge">.yml</code></strong> ending!</p>
<div class="language-sh highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c"># content of .rubocop</span>
<span class="nt">--server</span>
</code></pre></div></div>Crawl MediaMarkts Fundgrube using Babashka (and telegram bots)2022-11-06T00:00:00+00:002022-11-06T00:00:00+00:00repo://posts.collection/_posts/2022/2022-11-06-fundgrube.md<p>While browsing through my GitHub Feed, I stumbled across <a href="https://github.com/RomanNess/fundgrube-crawler">Roman Ness’ fundgrube-crawler</a> and thought this to be a nice little code exercise.</p>
<p>My <a href="https://github.com/simonneutert/fundgrube">fundgrube code on GitHub</a>.</p>
<h2 id="what-is-mediamarktsaturn-fundgube-and-what-is-the-problem-with-it">What is MediaMarkt/Saturn Fundgube and what is the problem with it?</h2>
<p>The Fundgrube is a collection of products in a sub-webshop you can visit in MediaMarkt/Saturn’s main webshop. The products listed there usually are reduced in price, reduced drastically. These products were maybe used for displaying in the shop or may have some cosmetic issues, like scratches. Here’s the twist: customers need to collect the product from the store, shipping is not an option and another customer might be quicker than you.</p>
<p>The “problem” with how the Fundgrube is offered as a webshop is the lack of filters or proper search tools. Just as they want you to come to the store to pick the product up, they want you to check the Fundgrube regularly (I guess).</p>
<p>The data/products API is public and when opening the dev tools in your browser, you will find a custom header, wanting you to send in your resume and apply for a job. Fair play, MediaMarkt and Saturn.</p>
<h2 id="my-script-my-babashka">My script, my babashka</h2>
<p>What I’ve come up with in an adrenalin rush, is not (yet) a script i would show my mother-in-law and brag about it…, but …, well …, it does the job 😅</p>
<p>My script crawls through MediaMarkts public <a href="https://www.mediamarkt.de/de/data/fundgrube">Fundgrube</a> API and posts new products on offer via a <a href="https://core.telegram.org/bots/faq#how-do-i-create-a-bot">telegram bot</a> in a secret channel for me and some friends.</p>
<h4 id="technicals">Technicals</h4>
<p>The tool is parameterised using ENV Vars, to iterate over the local product postings, then collecting unique items, aggregating the results. Before storing the data to disk, the last time created file is being copied and renamed, allowing the tool to detect changes. I decided to use Clojure’s <a href="https://github.com/edn-format/edn">edn data notation</a>.</p>
<p>Here’s an example what a post to telegram of a detected product entry would look like:</p>
<div class="language-text highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Markt: Wiesbaden-Hasengarten
Produkt: PHILIPS 58PUS8546/12 LED TV (Flat, 58 Zoll / 146 cm, UHD 4K, SMART TV, Ambilight, Android TV™ 10 (Q))
Preis: 700.00 €
Artikel 2734789
-
LED TV / 146cm/58Zoll / UHD 4K,
Ultra Resolution, Dolby Vision, HDR10+, Micro Dimming Pro, ISF-Farbmanagement,
2 x 10 W Full-Range-Lautsprecher,
Ausschalt-Timer, Lichtsensor, Bildabschaltung (bei Radiobetrieb), Eco-Modus
https://assets.mmsrg.com/is/166325/12975367df8e182e57044734f5165e190/c3/-/29336490249241a3925bcb97e235b830
</code></pre></div></div>
<p>What started as a quick experiment using httpie and jq, escalated quickly to an exercise in Clojure 🥰</p>I made my own little selfhostable flat-file notebook2022-10-29T00:00:00+00:002022-10-29T00:00:00+00:00repo://posts.collection/_posts/2022/2022-10-29-labradorite.md<p>Most Notetaking apps do either too much (Notion), too less (Apple Notes) or locked me in (Evernote).</p>
<p><img src="/images/2022/labradorite.jpg" width="50%" style=" display: block; margin-left: auto; margin-right: auto; width: 50%;" /></p>
<p>I wanted something that does <strong>just-enough</strong>™, in the spirit of a <em>flat file cms</em>. Let me try and wrap it up in a few bullets:</p>
<ul>
<li><strong>ownership</strong> of the notes (flat files)<br />
markdown, yaml, attachments all in one place in directories</li>
<li><strong>easy</strong> on the eyes/mind<br />
I don’t want to see all the old stuff I may not need anymore</li>
<li>a search that shows me <strong>snips</strong> in the notes<br />
not just the notes in a sidebar</li>
<li>a very simple <strong>tagging</strong> support</li>
<li>basic <strong>file uploads</strong><br />
keeping the originals side by side with notes</li>
</ul>
<p>I like to keep things simple, so I once again fell for <a href="https://roda.jeremyevans.net">Roda</a> as the Web Layer. <a href="https://github.com/baygeldin/tantiny">Tantiny</a> is the kicker for this project. I was curious, if I could come up with something useful quickly, not having to deal with a complicated setup or Postgres as a requirement.</p>
<p>You can find the <a href="https://github.com/simonneutert/labradorite-notebook">source code on GitHub</a>. Give it a spin and contribute if you are missing something.</p>