Log in

English posts

Yak, strangling PHP with Phoenix and NGINX

At the time of writing this blogpost, my site is written in PHP, and most of the code is many years old. One could call it a legacy application: I was not very good at organising code back when I started it, so it is a bit of a mess.

Like any legacy application, rewriting it would be costly. There are many features I don’t want to lose, but at the same time I don’t want to lock myself up and rewrite them all before I can show and use any work. This kind of ‘Big Bang release’ would eat up all my free time for the next months and my motivation probably won’t make it all the way through.

A Yak to shave

Yesterday I attended the first IndieWebCamp since ‘March 2020’ and I didn’t want to come empty handed. I forced myself to start a new project called Yak.

This is actually the fifth project called ‘Yak’ on my system, but it’s the first one I actually deployed. All the Yak-projects focus on a different part of the app, since there is always something you think you should do first before the rest can start.

The newest and deployed Yak consists of a new IndieAuth endpoint. I was able to shave off this part since IndieAuth tries to be decoupled from the rest of the site: its endpoint might even be on a different domain.

I didn’t go that route though: I decided to strangle.

The simple strangle

With ‘strangle’ I refer to the ‘Strangler Pattern’, after the now more friendly(?) Named term ’Strangler Fig Application’ by Martin Fowler. This plant grows around another tree, slowly taking over its shape and killing it in the process, so that eventually, all that remains is the fig.

In terms of Seblog.nl, I would like Yak to take over more and more features from the PHP-version, until finally Yak is my new CMS. I say ‘PHP-version’, be because yes, Yak is written in Elixir now. How would I go about taking over features then? My first attempt was this:

index index.html index.php;

location /auth {
    # ... proxy headers
    proxy_pass http://localhost:4001;

location / {
    try_files $uri $uri/ index.php$is_args$query_string;

location ~ \.php$ {
    fastcgi_pass unix:/var/run/php/php7.4-fpm.sock;
    fastcgi_index /var/www/seblog.nl/index.php;
    fastcgi_param SCRIPT_FILENAME $request_filename;
    fastcgi_param PATH_INFO $fastcgi_script_name;
    include fastcgi_params;

That’s a pretty normal PHP setup, but with one path as a proxy. Yak is listening to port 4001 and NGINX sends all traffic for /auth and sub-folders to it. (A firewall keeps visitors for 4001 from outside away, don’t worry.) This way I can have my IndieAuth endpoint be in a different application while still maintaining one domain to the outside.

But since this is my hobby project, I can get niftier.

Proxy all the things

The IndieAuth route is fairly well scoped and easily separated from the rest of the app. For creating posts via Micropub, I can also separate a route and then store the created posts in the same file folder (or database) since both apps run on the same machine. But not all features are that easily separated.

To make it myself easier to add new routes to Yak, I created the following monster:

index index.html;

location @yak {
    # ... proxy headers
    proxy_pass http://localhost:4001;
    proxy_intercept_errors on;
    error_page 404 = /index.php$is_args$query_string;
    error_page 502 = /index.php$is_args$query_string;

location / {
    try_files $uri @yak;

location ~ \.php$ {
    # same as previous PHP

Since Yak is pretty fast (the Phoenix framework logs its response times in microseconds instead of milliseconds), I decided to just route all traffic through Yak. If Yak has the answer, that answer will be the final answer. But all routes that get a 404 Not Found error from Yak, are given to PHP afterwards.

This way the old site still functions: PHP is not even aware that someone has looked at the request before it. I also added the 502 Bad Gateway just in case Yak is down, which I expect only to happen for brief moments during big configuration changing deployments.

A few details to note: it all starts by adding @yak as a final location to try_files. Only the last location can be a virtual one, otherwise I would have put PHP directly in that list. I did remove the $uri/ and removed index.php from the index directive at the top: without this the homepage was not delivered to Yak.

The other crucial bit is the proxy_intercept_errors on, because that traps the errors in the proxy and enables you to add cases for certain errors. With error_page 404 = /index.php$is_args$query_string I send them to PHP. Other errors from Yak, like 401 and 403, are still just rendered as they came from Yak.

Fixing my POST to Micropub

While the above does work for most of my site, it broke my Webmentions and Micropub. (Yes, I tried posting this a few hours ago.) The previously described method makes all requests in PHP come in as GET, which makes sense for rendering an error page.

After a lot of trial and error, reading NGINX documentation and even trying to get Yak to proxy, I decided to localise the few POST requests I needed to be PHP and added those explicit in NGINX.

location /webmention {
    try_files $uri $uri/ /index.php$is_args$args;
location /micropub {
    try_files $uri $uri/ /index.php$is_args$args;

Once I moved these endpoints to Yak, I can remove them from my NGINX.

Homebrew Website Club London

So, I am not in London and I am not even in the precise timezone as London, but since The Situation is still keeping us home, I got to attend Homebrew Website Club London.

It was mostly just some chatting about smoke detectors, automated blinds and visits to city water reservoirs, as one does on a HWC. We had a few on-topic points as well.

I told about the upcoming birthday of Seblog.nl, tomorrow, which got us down the path of looking up old versions of websites. Much is saved, but many things are lost as well. One thing we came to: if you are starting to code your own website, please learn how to use version control as soon as possible. I (and others) have lost old versions of our sites because we kept overwriting the old files with new changes. If I had discovered Git (or any other version control) earlier, I would have had the oldest versions still.

As a note to myself: I should read Peter's article that he mentioned, which is about this 'content archeology': bringing back old home-pages. I doubt I have enough time to excavate my own first version of Seblog.nl before tomorrow.

Calum also showed his new bookshelf page. It reminded me of my own page called /bieb (short Dutch for 'library'), and now I want to revisit that page as well. The past month, I've been playing around with Obsidian and this would be one of those places where my site could integrate with it. (Both Obsidian and the current iteration of my site run on raw text files.)

I also shared some of my plans around this integration but I am not ready to share those here. (Most of my projects become vaporware, sadly.) However, I feel encouraged that my idea was not totally a bad idea. (Only slightly.) That's why I like going to HWC's: they spark ideas and / or bring them further.

als antwoord op Ako Suzuki

The reason I follow your lessons is exactly because they are in Japanese!

  1. You just learn more Japanese by using it.

  2. As a Dutch person, I’m okay with English, but I’m not native and some words just better translate to Dutch. (深夜, ‘diep in de nacht’; まずい, ‘vies’)
als antwoord op Amber Wilson

It was about Zelda, I believe, and I kept it on a floppy instead of a server. It had a “secret page”, which was linked from one of the letters of a page header. It was a pretty obvious link though: I had no idea how to get rid of the underscore with just HTML.

VIM productivity is a lie: all the time you save with your fancy commands and macros, you loose again on bragging about them to your coworkers.

Due to The Situation I only listened to Dutch podcasts for a while. Today I went for a bike ride and I wanted to listen to some dev-related talk again, and yep, I really thought someone was ringing at me when the @_bikeshed started.

Let’s talk about ‘de’ and ‘het’

Speakers of Dutch may ignore this post. Learners of Dutch might be interested.

I offered Henrique to answer questions about Dutch, because I like learning languages myself, and know a bit about it, since it’s my native language and I’ve studied it. In the past I wanted to start a learning resource for it, but I figured I can just start on my own site first.

The first question is about ‘de’ and ‘het’, the two articles in Dutch. They can be confusing. And I hate to break it: I can clear some of the confusion, but they will always be hard to learn.

Genders as groups of nouns

Articles are weird things. Dutch has two, English has one, French and Spanish also have two, but German even has three. In French and Spanish, la is considered to be feminine, and le/el is considered to be masculine. One might ask: which one is masculine in Dutch, de or het? Unfortunately, the answer is that de is both masculine and feminine.

If we look at German, we have der, die and das. Here, der is masculine, die is feminine and das is neither. In the same way in Dutch: de is der and die combined, and het is das. We call it ‘onzijdig’, having no side.

I have never really understood the whole masculine/feminine thing. There is nothing masculine in the words itself. It is just that we have divided our nouns into groups (sometimes two groups, sometimes three or even more). We then let other words ‘react’ to these groups, which helps understanding the sentence if it contains a lot of words. Over time, linguists have put labels on these groups of words, and they used ‘masculine’ and ‘feminine’ to denote them, but they could have been ‘red’ and ‘green’ too.

When learning another language, these labels actually don’t make sense. Some words that are masculine in one language, end up feminine in the other. The same for German and Dutch: a word being das in German probably has a higher chance of being a het-word in Dutch, but there are no guarantees at all. (This is why I never dare to speak German, although I can understand it when written or spoken slowly.)

Is it ‘de’ or is it ‘het’?

Spanish has this nice rule where a lot of nouns end in either an -a or an -o sound. There are endings in German that will predict the gender of a word too. They are not really relyable though, and for Dutch, I don’t know of such rules. To be honest: you just have to ‘hear’ it.

This is the most horrible advise Dutch people will give with regards to de and het: I just hear it. There is nothing in the words itself that gives away why it will ‘sound good’. What Dutch people mean, is that they always hear a certain word – like ‘hond’ (dog) or ‘paard’ (horse) – in combination with a certain article (‘de hond’, ‘het paard’). We hear it so often, that we get used to it.

If I would have to give an advice for learning it: repeat the words. Hearing the combinations ‘de hond’ and ‘het paard’ often, make the connection. Try to feel that it ‘sounds good’ to hear ‘het dorp’ and ‘een bruin paard’. There is nothing more to it.

But I hope this helps: most words are de-words, but among common words, there are a lot of het-words. Said differently: try to focus on the het-words, for there are a lot of common words among them. Once you know a lot of common words, chances are that this new word you don’t know yet is just a de-word.

The one exception to all of this, are diminutive words, on which I might blog later. If it ends with -tje, -pje, -kje or -je in general, it’s probably a ‘verkleinwoordje’, and thus a het-word. So: ‘de hond’, ‘het hondje’; ‘het paard’, ‘het paardje’; ‘de piano’, ‘het pianootje’. If it got small, turned into a het-word.

The changing contexts

Earlier I said that other words ‘react’ to the gender of the noun. German has a reputation for this, and luckily Dutch lost almost all of it’s conjugations in this part. Except for one: sometimes, the ending -e gets lost for het-words. Let me give you some examples:

  • De hond is bruin. De bruine hond.
    The dog is brown. The brown dog.
  • Het paard is wit. Het witte paard.
    The horse is white. The white horse.
  • Een hond is grappig. Een grappige hond.
    A dog is funny. A funny dog.
  • Een paard is kalm. Een kalm paard.
    A horse is calm. A calm horse.

Note that in the last example, there is no extra -e on the adjective. This has nothing to do with the word ‘kalm’, but everything to do with the combination of ‘een’ and the het-word ‘paard’. In the context of een, the extra -e disappears for het-words, just to mess with you. (The difference between de/het and een is exactly the same as the and a/an in English.)

To add examples:

  • De kalme hond.
  • Een grappig paard.
  • Een wit paard.
  • Een bruin schaap. Het bruine schaap.
  • Een rode hond. De rode hond.
  • Een rood hondje. Het rode hondje.

Here, also, I would say that the only way out is to try and ‘feel’ that it ‘sounds good’ this way. But with these rules, you can at least find the other form that should ‘sound good’ based on the one pair you know that ‘sounds good’.

Hope this helps. I promise, Swedish is worse when it comes to nouns.

On Private Commenting Systems

Jan-Lukas wrote about an article by Matt from Write.as called Towards a Commenting System. The article describes a commenting system with two flavour: private and public. For private comments, an e-mail to the author is used; for public comments, one is prompted to publish the comment on their own space first and then notify the original post. It feels very IndieWeb friendly.

Then, Jan-Lukas points out his own site already does this with webmention (as does mine), and that he also has a contact form, which people could use to reply private (at the time of writing I have no contact form).

I like the idea of private comments taking another route than public comments. Just having a contact page is not the same though: to complete the idea you can link it with a call-to-action underneath your posts. Let’s not have illusions here: most people will probably not read my posts on my site but in their reader or some other syndicated copy. But, it would give a nice UX for those on my site.

It also reminds me of how stories on Instagram work. There is a text box underneath it, which the user can tab to type a message. This message is then sent as a direct message to the creator of the story, and not visible to anyone but the creator and the commenter. It seems to work in that context: I do reply to friends in that way sometimes, because it feels very personal.

Another point is that this keeps private comments easier to implement for some and actually possible for those with static generated sites. My site has a way for visitors to log in, and I can build some form of private comments in that way. It is, however, way more work to build and maintain a site that does this, and not everyone is willing to do so. Doing private comments via a different channel makes it easier to have them.

The flip-side is that private comments cannot be shared among a group in this way. If you open a post to a certain circle of friends on, say, Facebook, all those friends can comment and also comment on each other’s comments. This kind of interaction is very hard to do, though, if you don’t have the luxury of a central service that guards access to all the posts — like Facebook does.

The conversation is also more likely to be ephemeral, for there are only two readers, both responsible for keeping their copies, with no help of, say, the Internet Archive.

I remember a moment at IndieWebCamp Brighton when a session about private posts was about to start. Jeremy Keith walked out of the room while making a comment that he didn’t see private pages as something the Web needed. This does not mean I can’t have them, but it did make me think about why I want them and what they would mean to the Web and the world.

By putting the private comments on a separate channel, you are also removing them from the Web. This makes the Web a place for open and public conversations again. (Again: one could argue that thing on the ‘Private Web’ are not on the Web either.) The last few months I’ve been reading more blogs and I must say I really enjoy that open Web.

The separation of private comments creates a clearer boundary between the open and the private, and maybe that’s a good thing. It makes an easier question: it’s harder to answer “which people should be able to read this post?”, than it is to answer “does this concern only me and the author, or could there possibly be someone out there who’s interested?”.

No real plans for removal yet, but I keep being torn about private posts.

RSS is not dead you know (nor is Atom)

Quite a while I wrote about building a social reader[^ Then called IndieWeb reader, but social reader is the better term.], but these days I have to admit it went nowhere. The biggest problem with it being that I myself don’t really use any reader to consume stuff: I was not used to keeping up with blogs, I only used to mindlessly scroll through Instagram. These days I got the Twitter app back on my phone again, so I make my scrolls there too.

What does seem to work over the past two months or so, is that I occasionally look at the NetNewsWire Mac app, which I pinned to my dock. Seeing the icon makes me click on it, and this way I do read or skim blogs I follow. I organised stuff by social distance, or something close to it, like Ton suggested. I notice that it makes me sad some people I want to follow as a person are not available via RSS/Atom.

The experience on NetNewsWire is not quite the way I want it, but I figured it is better to use an imperfect setup than no setup at all. I really enjoy being in the loop with content I actually care about, sorted by how much I care about it (thanks Ton). Maybe from here I can improve things (including but not limited to my own RSS format).

NetNewsWire just released an iOS app, so I have to check that out too.

Random generators again

When I started coding around 2005, two questions kept me going: how on earth does a .php file on a server produce a different page if you append ?p=22 to it’s URL; and how do I make my own random generator? The magic of the former now disappeared behind pretty URL schemas (also: there isn’t much depth to it, it turned out), but the latter still catches me from time to time.

Back then, I was intrigued by the site Seventh Sanctum, which was and still is home to many generators. Want a wacky gadget, a quick name, or a wrestling move? One click of a button and you have 15. Heck, I wanted to give three examples so I clicked the ‘random generator’ button three times and it gave me these, like a generator that generates generators.

I made several generators myself, most notably: Randomon, an unfinished and thus empty picture with the height and name of a Poké/Digimon-like creature; the Stamboomgenerator, which draws a wacky family tree with Dutch names and surnames; and various attempts to generate artificial languages. Since I did not know about version control back then I probably lost a lot of it.

Drawing the new city

A few weeks ago, our Dungeon Master Mike showed us the map of the city our characters had just arrived at. It looked awesome: he had drawn roads, walls and it was filled with boxes that represent houses. He’s got time too much, I thought. Also, wouldn’t it be cool if you had a generator that just generates a map like that?

I had been playing with Rust and it’s modular game engine Piston, and you thought: wow, if Rust is really that fast, I can generate a lot of stuff. It ended with me, turning the Hello, World!-example of a spinning red cube into a red cube that could be panned around, with zoom and everything. It’s almost Google Maps, I imagined. Now I just have to make the map.

It was then that the Duck lead me to an article by Amit Patel about generating maps from noise. And then that lead me to his brilliant explanation about noise and just all the rest of his site.

See, back in the day, I quickly discovered that random is both your friend and your enemy in the game of generators. Too much randomness makes certain properties unbelievable. Also, I wasn’t very good at writing and thinking about code back then. The following is literally how the Stamboomgenerator chose how many children a certain family member would have:

$kinderen = array(0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,3,3,3,3,3,3,3,3,3,3,4,4,4,4,4,4,4,5,5,5,5,5,5,6,6,6,6,6,7,7,7,7,8,8,8,9,9,10);
$kinderen = $kinderen[rand(0,count($kinderen)-1)];

I tweaked it by hand, by just running it a lot of times, feeling whether or not the proportion would make sense. There is zero math behind it, I’ve studied Dutch literature, you know.

Everything has already been done

Studying the Red Blob Games site, and getting extra info about the terms on other sites and Youtube, I discovered so much more world (and math) behind these generators. Turns out it’s also called ‘procedural generation’, which gives better search results. Everything already exists, but you need the proper name to Google it.

I don’t recall if I found out about Perlin noise before or after finding the Red Blob Games site, but while reading the articles I keep having “oh but you could…”s and “if you just…”s, which are most of the time resolved by just another article that shows you how to do it. And they all come with an interactive example with sliders for you to adjust as to understand it better. It is a crazy treasure to find.

The whole thing made me also wander off into the world of 3D rendering, exploring OpenGL and linear algebra. It resulted in last weekend’s fiddling with Blender and a donut. This is a totally different topic, but that’s also a field where a lot of knowledge is shared around. There is so much to learn.

Generating my own world

I kept thinking about how to combine all the elements of map making with noise with other approaches, that would give me plate tectonics. I would then like to adjust the scale of generation from millions of years to just years and play out a simulation of human influence on that map, as to finally arrive at a generated city with an actual history.

For the history to play out, I need some form of grid system to divide the planet into manageable chunks. Luckily Amit has written about that too. But unlike me, he actually reads mathematics papers about this stuff, and thus knows that it’s impossible to divide a sphere into equal hexagons and stuff. (You need 12 pentagons.)

And then of course, he also did it: he made a thing that makes planets with tectonics. Including an example that renders in your browser, with slider to change some parameters. It is both so cool and so intimidating.

It’s not that I’m discouraged by the knowledge that someone else has already done things I wanted to do. It’s impossible to be original (and also: it’s impossible to exactly copy). But the intimidating part is all the math behind his stuff, and the complete lack of it on my side.

Still, with this knowledge I might just take some shortcuts with the world and go for the simulation path. It has been done, I know, that was what lead me to Perlin noise. There are just so many fascinating aspects of this topic.

Writing this blogpost probably makes it less likely for things happen, but I owe it to people like Amit to also think out loud sometimes, to share what I have found. Maybe more about this in the future.

Oh and by the way, Mike did not draw that map himself, as I found out. ’Ik heb wel wat beters te doen,’ he said laughingly.

als antwoord op Matthias Ott

‪Strava and an Apple Watch for me. But: the Strava app is no good at all. I use the native Apple Watch Workouts app, which is quite good, and an app called HealthFit to export from HealthKit to Strava (and maybe my own site, if only I found time).‬

I’m in the train between Leiden and The Hague and for almost the full ten minutes of that trip, two trains ride next to each other at about the same speed. Very cinematic to see. (Although I admit this video has too much reflection to be truely cinematic.)

[laravel/ideas] Calling a Pipeline multiple times

So, currently the Illuminate\Pipeline\Pipeline is used for Route Middleware and Job Middleware. In both cases, the Pipeline is used once: you new up an Instance and pass it the Request or Job, then use the result. It is clearly optimized for that use case.

I have a use-case in which I would like to call a single Pipeline (consisting of the same pipes) over and over again. When I used the Pipeline as is, I was surprised to learn that each time I run the pipeline with ->then(), I would get new instances of my Middleware. An example:

$pipeline = resolve(Pipeline::class)

$handle = fopen('logs.txt', 'r');
while ($line = fgets($handle)) {
    $result = $pipeline
        ->then(fn($line) => $line);
    // ...

This will create new instances of all $pipes for each line in the file. This is undesirable if you want to do something with the lines based on a state of the pipe. As a silly example:

class LogRowCounter
    protected $counter = 0;

    public function handle($line, $next)

        return $next($this->counter . ' - ' . $line);

Apart from resolving all the $pipes, a lot of checks are also done while running the pipeline, meaning it happens in the while loop of my example. Some of those things could be done beforehand.

Reusable Pipeline

To make the Pipeline reusable, I would propose a new method called prepare(). (I was also thinking about compile(), but ‘prepare’ is used in some protected methods of Pipeline already.) It would extract running the array_reduce() from then() and return the resulting closure, and then() can call it.

    public function prepare(Closure $destination)
        $pipeline = array_reduce(
            array_reverse($this->pipes()), $this->carry(), $this->prepareDestination($destination)

        return $pipeline;

    public function then(Closure $destination)
        $pipeline = $this->prepare($destination);

        return $pipeline($this->passable);

This would still resolve all the pipes on ‘runtime’ though, because of how the closures are structured. Here is a new version of carry(), which resolves the pipes outside of the returned closure. The outer closure returned is meant for array_reduce(), like the current version; didn't want to refactor that.

    protected function carry()
        return function ($stack, $pipe) {
            if (is_string($pipe)) {
                [$name, $parameters] = $this->parsePipeString($pipe);

                $pipe = $this->getContainer()->make($name);
            } else {
                $parameters = [];

            return function ($passable) use ($stack, $pipe, $parameters) {
                try {
                    if (is_callable($pipe)) {
                        return $pipe($passable, $stack);

                    $parameters = array_merge([$passable, $stack], $parameters);

                    $carry = method_exists($pipe, $this->method)
                                    ? $pipe->{$this->method}(...$parameters)
                                    : $pipe(...$parameters);

                    return $this->handleCarry($carry);
                } catch (Exception $e) {
                    return $this->handleException($passable, $e);
                } catch (Throwable $e) {
                    return $this->handleException($passable, new FatalThrowableError($e));

All Pipeline tests pass, and the following new test would pass too:

    public function testPipelinePrepared()
        $pipeline = (new Pipeline(new Container))
                    ->prepare(function ($piped) {
                        return $piped;

        $result = $pipeline('foo');

        $this->assertSame('foo', $result);
        $this->assertSame('foo', $_SERVER['__test.pipe.one']);
        $this->assertSame([], $_SERVER['__test.pipe.state']);

        $result = $pipeline('bar');

        $this->assertSame('bar', $result);
        $this->assertSame('bar', $_SERVER['__test.pipe.one']);
        $this->assertSame([0, 1], $_SERVER['__test.pipe.state']);


class PipelineTestStatefulPipe
    protected $count = 0;

    public function handle($piped, $next)
        $_SERVER['__test.pipe.state'][] = $this->count++;

        return $next($piped);

Things to note

First off: I’ve taken the liberty to rename the ! is_object($pipe) check to is_string($pipe), because the very next thing we do is call $this->parsePipeString($pipe) with it.

Because of the way the closure is bound, the following code would work, and it seems like I can put it to use, but it looks wonky and weird too:

$filename = 'logs.txt';
$pipeline = resolve(Pipeline::class)

$run = $pipeline->prepare(fn($line) => $line);

// Call all `init($filename, $next)` methods

// Set method back to 'handle'

$handle = fopen($filename, 'r');
while ($line = fgets($handle)) {
    // Call all `handle($line, $next)` methods
    $result = $run($line);

    // ...

… that syntax makes me feel like Pipeline should store the Closure as a property somewhere, so prepare() returns $this, and then add an __invoke() or run() to Pipeline, so you can at least use the same variable instead of $pipeline and $run.

Alternative solutions

While we could do the above, I also have a different solution: singletons in the Container.

I originally rejected this (hence the work above) because I don’t want my stateful pipes to be global singletons. I want the pipes to belong to the Pipeline: if the Pipeline goes, so go the references to the pipes.

Think for example of the above file-handling example, but as a queued Job: I don’t want the state of the pipes from one run to be around when I process the next log-file.

But: this last point can also be resolved by creating a new Container instance per Job, and giving that to the Pipeline while constructing it. You can bind singletons into that separate container, which will belong to the pipeline.

Note that in that case – if we don’t change the Pipeline class – you’re still making the calls to the Container on ‘runtime’, which is not optimal. On the other hand, it might not be a very big deal.

I don't really like the way I have to new-up or resolve the Pipeline in this case, but it is doable.

I’d be happy to work this into a PR, but I thought this might be a better start, especially since I’m still debating whether or not I got a point here.

Thanks for reading, I’m curious about your thoughts.

Really liking the arrow functions in PHP 7.4 so far! Unfortunately already found one missing feature: you can't throw Exceptions from them. The following results in a 'unexpected T_THROW' syntax error:

fn() => throw new Exception('nope');

Would've cleaned up my test, but alas.

Introducing dark mode

Like a few others this IndieWebCamp I added a dark mode to my website.

With iOS 13 having a dark mode, which toggles with sunrise and sunset if you want it to, I all of the sudden like to have it on. And yesterday in the train, I noticed that my site felt kind of bright.

So this morning I hacked it together in the train back to Amsterdam. I went the dirty way: just have one media-query for determining dark mode, and then target a lot of elements and classes inside that, and set their color to be something different.

The actual implementation was not that hard (@media (prefers-color-scheme: dark)), but I spent the most time of the traintrip figuring out which darker grayscales should replace the brighter grayscales.

One thing to note here was the tip by Steve Schoger I remembered reading: dark modes are not about just inverting colors. Important elements in your UI should still be brighter than others.

Luckily, I don’t have a very complex UI on my weblog, but it’s worth to note that I took some time to make my month overview pages look nice in both modes. Feel free to compare the two.

Note that I currently don’t have a switch on my site to make you choose, but if your browser / OS tell my site which mode it’s in, my site will adapt to that.

On the topic of scoring in NES Tetris, PAL vs. NTSC

tl;dr: This is nerding out on Tetris scores, and quite frankly, I don't think the conclusion holds. But there are nice tables along the way.

I've been watching Classic Tetris for a few months now, and as a result, I'm also playing it. I am noticeably better after I watched some pro's scoring Tetris after Tetris at level 19, so I keep this pace of watching, playing, watching playing.

There is a problem though: at first I deemed playing at level 18 myself impossible, it's just too fast. Then I noticed how my mind could keep up with the players on the screen, but not with the pieces falling on my screen. Of course, I need more practice, but I have an excuse: I live in Europe.

Turns out old European TVs have a different refresh-rate than Americans. Therefore Tetris on the European PAL systems gets faster in earlier levels than the American NTSC systems. There is a nice article by Tetris Finland about this and other differences between the two versions.

Despite the faster speeds of the PAL version, I found no evidence of the scoring system being different. This means the PAL version is just harder, you just get less points for your hard work. (Yea, I will come back at this.)

I am playing Tetris on a real NES, brought ±15 years ago on a flee market at the local community playground for the insane price of 10 euros. The cartridge of Tetris has cost me more. But to compare myself with the pro's on screen I would have to spend a bit more to import an American NES.

Isn't there a way to calculate the NTSC score based on a PAL score?

Yea, is there a way to calculate NTSC score?

My current, not too impressive high-score on PAL is 174_731 (and yes I will use underscores here). How much is that worth on NTSC?

My first attempt was to look at the maximum scores. Players have been known to 'max out' the game on NTSC. A max out score is a score that is higher than 999_999, at which point the score will freeze on those six nines.

On the PAL version, however, we still have a world record, which is currently set to 758_360 by Joseph Saelee. This video is also a demonstration of how much harder the PAL game gets. (Although Joseph has a talent of making Tetris look easy on all versions.)

With some tweaks in the software you can get Tetris to report past 999_999, so higher scores are known. But for simplicity, let's take the world record and equate it with a NTSC max out score. This gives us ntsc = pal / 758_360 * 999_999, which will give me a NTSC score of 230_406.

But is that accurate?

It's always comparing apples with oranges, but I feel like this formula is a bit too loose. Let's bring in the information of the aforementioned article of Tetris Finland, which gives a nice table that compares the different frame rates, and calculates per level the time it takes, in seconds, to drop 20 spaces. Here is a table with that data:

9 1.996 sec 1.999 sec
10-12 1.663 sec 1.599 sec
13-15 1.331 sec 1.199 sec
16-18 0.998 sec 0.7998 sec
19-28 0.6655 sec 0.39994 sec
29+ 0.33278 sec as above

Now, the scoring stays the same in both versions, but changes per level. Let's bring that down to a single number per level, let's agree on a perfect score.

The best way of scoring points is to only score tetrises (clear 4 lines at once). Each tetromino is made up of 4 blocks, and the board is 10 blocks wide. A perfect score would be if the Random Number Generator (RNG) gives us only lines and we can stand them up over the full width of the board, resulting in a tetris and an all-clear board each 10 pieces. This is not really achievable, but this would be the best score possible.

According to the Tetris Wiki we get 1200 * (level + 1) for each tetris, and scores are calculated based on the level after the line clear. Let's ignore push-down points. Let's also say that if the drop speed goes below 0.5 sec for 20 spaces the game is over (this makes levels 29 for NTSC and 19 for PAL the kill screens).

Then there is the concept op the start level. When you start at level 1, you go up to level 2 after 10 lines. But when you start at level 18, you get 130 lines before going to level 19, after which you get a new level ever 10 lines again. The Tetris Wiki gives the formula of "(startLevel × 10 + 10) or max(100, (startLevel × 10 - 50)) lines, whatever comes first". This means 100 lines for levels 9-15, then 110, 120 and 130 for 16, 17 and 18.

So I wrote this odd-looking piece of Javascript (because it's my day off and I can do whatever) to list out the various perfect scores for the various level-starts.

const killscreen = 19 // or 29

tetrisScore = level => 1200 * (level + 1)

startlevelLines = level => Math.min(
  (level * 10 + 10),
  Math.max(100, (level * 10 - 50))

scoreForLevel = (level, lines) => ({
  score: (Math.floor(lines / 4) * tetrisScore(level)),
  restlines: lines % 4

restlevelScore = (level, rest) => {
  if (level >= killscreen) return tetrisScore(level)
  const {score, restlines} = scoreForLevel(level, 10 + rest)
  return score + restlevelScore(level + 1, restlines)

perfectScore = level => {
  const {score, restlines} = scoreForLevel(level, startlevelLines(level))
  return score + restlevelScore(level + 1, restlines)

const levels = [...Array(killscreen).keys()]
levels.map(level => console.log(`| ${level} | ${perfectScore(level)}`))

I was kind of surprised by the results: as you can see, the world record by Joseph is higher than the perfect score. This is because he had a 100% tetris rate, burned one line in level 18 for 760 points, then tetrised into level 19, and then made a few singles and doubles in level 19. My definition of 'kill screen' seems too strict, but at least it's even on both sides of the table.

Start level NTSC perfect PAL perfect
0 1_332_000 588_000
1 1_334_400 590_400
2 1_340_400 596_400
3 1_348_800 604_800
4 1_360_800 616_800
5 1_375_200 631_200
6 1_393_200 649_200
7 1_413_600 669_600
8 1_437_600 693_600
9 1_464_000 720_000
10 1_478_400 728_400
11 1_454_400 710_400
12 1_462_800 712_800
13 1_432_800 688_800
14 1_435_200 685_200
15 1_399_200 655_200
16 1_429_200 685_200
17 1_461_600 717_600
18 1_497_600 753_600
19 1_536_000
20 1_578_000
21 1_622_400
22 1_670_400
23 1_720_800
24 1_774_800
25 1_831_200
26 1_891_200
27 1_953_600
28 2_019_600

Given that my high score was a level-5-start score of 174_731, I should use ntsc = pal / 631_200 * 1_375_200, giving me a NTSC score of 380_687, a lot more than my previously calculated 230_406. But is this fair now?

Score per second

The table above is not by any means a fair comparison. It does account for the higher scoring potential in NTSC, but it still compares levels regardless of the speed they are played at. Let's combine the previous two tables into one.

So in the PAL version, you get less points, but also less time to think about how to score them. In that regard, the PAL version is harder, and the points should be worth more.

In the next table, I've taken the perfect score per level, and divided it by the thinking time you have for it (the seconds of the earlier table times 240, the number of lines before transition you get when you start in level 29). The result is a series of numbers that say how 'easy' it was to get that number of points.

Start level NTSC score sec / 20 lines modifier
9 1_464_000 1.996 3.0561122244
10 1_478_400 1.663 3.7041491281
11 1_454_400 1.663 3.6440168370
12 1_462_800 1.663 3.6650631389
13 1_432_800 1.331 4.4853493614
14 1_435_200 1.331 4.4928625094
15 1_399_200 1.331 4.3801652893
16 1_429_200 0.998 5.9669338677
17 1_461_600 0.998 6.1022044088
18 1_497_600 0.998 6.2525050100
19 1_536_000 0.665 9.6240601504
20 1_578_000 0.665 9.8872180451
21 1_622_400 0.665 10.1654135338
22 1_670_400 0.665 10.4661654135
23 1_720_800 0.665 10.7819548872
24 1_774_800 0.665 11.1203007519
25 1_831_200 0.665 11.4736842105
26 1_891_200 0.665 11.8496240602
27 1_953_600 0.665 12.2406015038
28 2_019_600 0.665 12.6541353383
Start level PAL score sec / 20 lines modifier
9 720_000 1.999 1.5007503752
10 728_400 1.559 1.9467607441
11 710_400 1.559 1.8986529827
12 712_800 1.559 1.9050673509
13 688_800 1.199 2.3936613845
14 685_200 1.199 2.3811509591
15 655_200 1.199 2.2768974145
16 685_200 0.799 3.5732165207
17 717_600 0.799 3.7421777222
18 753_600 0.799 3.9299123905

This would say an NTSC max-out on a level 18 start is 999_999 / 6.25250501, so has a difficulty of 159_935. The world record on PAL is 758_360 / 3.9299123905, so has a difficulty of 192_971. To me this feels believable, because Joseph has had many beyond-max-out games. We don't know their scores, because the game does not tell us, but their difficulty might also approach 200k.

And my score? Well, as I mentioned somewhere it was a level-5-start score, and as you can see in the first table: there is not much difference between level 9 on PAL and NTSC, and level 5 is not even listed. I should really stop whining and just practice more.

Apples and oranges

If you actually read the article by Tetris Finland you already would know: the speed is not the only difference. There are also differences in the way DAS works in PAL vs NTSC. The article concludes that these games are therefore very different, so different that you might consider them entirely different titles, despite looking the same.

It was fun figuring out these numbers and formula's, but in the end, you really can't compare the two. Gotta fix myself an NTSC somewhere if I want to compare my progress to the pro's.

Note that this difficulty number is a thing I invented in this post, and it is in itself not comparable to Tetris scores. But you can compare it to other difficulty numbers, if you calculate them the same way. Actually, I don't really think this whole thing is reliable at all, but I didn't want to throw a whole morning of calculating and writing out of the window. Some of the tables bear more value than others.

Meer laden