Log in
Seblog.nl

Seblog.nl

Days 1 to 10 of #100daysofindieweb

I’ve been doing this #100daysofindieweb thing for 10 days now, so I thought it was time for a summary post. Here are the things I've been working on:

Presentation

I fixed how things look here on my site quite a lot. On Day 1 I fixed how my reposts look. They are not longer just a url, but now actually show the original post, which I stored during the Twitter dump. I don’t do an automated grab of the external page yet!

I also changed now my RSVPs look on Day 7. First they where just a reply post, with a weird textual representation of ‘yes’ or ‘no’, but now they have icons.

On Day 8 I launched a ‘new post type’. In a way, this is just an different presentation of the posts I write for the purpose of writing them. I now only show the word count on those posts, not the text, because the actual text is less important.

Webmentions plugin

On Day 3, I fixed some bugs regarding relative urls, not only in my own Webmentions plugin, but also in the Kirby Toolkit. It eventually led to a new test on Webmention.rocks.

Today I also added an auto-archive function to my plugin, so it triggers archival copies for any url I mention.

Importing posts

I spend Day 5 and Day 6 importing old posts, which was actually quite hard and is still not really finished. I’m quite picky about how things should look, and they don’t look very good at the moment. There are also a lot of duplicate posts, because I imported my Twitter, and I used to link to my blog on Twitter. I still need to fix these things, but since I already spend two days on this, I postpone it some more.

Markup and headers

On Day 2, I marked up my deleted posts. If a post has a dt-deleted in the past, it returns a 410 Gone header on it’s own page, and shows up as a hidden tombstone in the feed.

And on Day 4 I changed the HTTP header of posts with a dt-published in the future to 404 Not found. Together with hiding them from the feed, this makes scheduled posts.

Finally I fiddled a bit with my Microformats on Day 9, so my site comes out better on Indiecards.


All in all it feels like a productive first 10 days! Only 9 of those sets to go :)

Day 10: auto-archive

Yesterday Tantek encouraged people to trigger an archive in the Internet Archive for every url they mention. It is not that hard to do.

So today I added an auto-archive function to my Kirby webmentions plugin, which you can find here. I’m not aware of anyone using it beside myself, but it is set up more or less so that everyone with a Kirby site can use it. (I am planning some breaking changes though.)

I guess, by posting this to my blog, I trigger an archive to a page describing how to trigger an archive. How meta.

Day 9: invisible things and indiecards

Today I did some invisible things. The first does not count for #100daysofindieweb, because it’s too invisible, but the second is valid, I think.


My blog runs on Kirby, which uses .txt files in a folder structure. In order to show you list posts, my server looks at several .txt files. The more posts I post, the more .txt files my server has to open. It all held up quite a while, but since my Twitter import my site contains 8000 pages.

This was not big of a problem for the main page, because I told Kirby to look at the newest folders first, and stop after 20 posts. (They are sorted by date, after all.) But when you want to view only the blogposts, or only posts with a photo, things became slower, because it had to look further in the past. Not to mention what would happen if you ask for a category that does not exist. With 8000 posts, my server would say no.

So I indexed all my posts with a database, a little while ago, and that seemed fine. The only problem was that the database is a pain to maintain. I did add pages to the database when I posted a post via Micropub, but when I edited pages via FTP, I needed to put ?reindex=1 behind the url to trigger a database update for that page. Sometimes you forget that kind of stuff.

Anyway. Today I made a system where entries are cached. I don’t cache the whole page, I just cache the h-entry, the part in this white block. The page now checks wether a .html version of the entry exists, and also checks the timestamps on the .html, the .txt and the entry.php snippet. If the .html turns out to be current, it shows the .html, and if not, it generates a new .html file and shows that new one.

With this new event, I reindex when the .txt has been updated. I still have to visit the page to trigger a reindex, but it’s much more smooth and almost goes without thinking. I’m very happy with how things turned out.

Unfortunately, all of this is not visible, and I haven’t opensourced this part either. And that’s not how things work in this challenge.


So, I needed to do something more visible for #100daysofindieweb. Yesterday I read about Kevin Marks’s Indiecards, and my site turned out weird on them. The main problem was that Indiecards look for the first h-* element on a page. Before today, the first h-* element on my homepage was a h-feed.

I fixed that today: my homepage now first gives a h-card. My h-entry pages still use that ‘same’ h-card as a p-author. Only my feeds now have a u-author set to /, which translates to https://seblog.nl/, which translates to the h-card on my homepage. More people do this, so I should be fine.

Day 8: the experimental post type ‘wrote’

Yesterday I admitted on IRC that I have quit the #100dagen500woorden part of my challenge. I already stated some doubts but Tantek added me to the wiki anyway.

I felt bad for giving up (so early!) but I felt even worse about posting strange unfinished texts on my site. That’s why I quit: the challenge didn’t feel good.

Today I thought about it some more. I needed a new post-type, where I only post how many words I wrote in a day, not the actual words I wrote.

Compare it with NaNoWriMo, where thousands of people write a novel of 50,000 words in the month of November. The actual text you write in that month does not really matter. The only thing that matters is writing a certain amount of words.

I never participated in NaNoWriMo, so I have no data to export in this format, but I can actually participate this year without having to put my data into NaNoWriMo.org.


Now for the implementation details. I have the following .txt-file on my development server:

This translates to the following public post on my blog:

As you can see, Seblog calculates the amount of words for me, so the only thing I have to do, is write a text and put it under the wrote field. I mark up the number of words with a p-words class, which is probably not consumed by anyone, but hey, I publish this kind of data now.

Now I have no excuse anymore to pick up some writing again.


Last but not least I want to coin the term ‘iceberg post’. A ‘wrote’ in the way I implemented it is an example of an iceberg post: there is a public part and a non-public part to it. It’s just a case of Partial Page Privacy, but I think the term ‘iceberg post’ has a nice ring to it.

Day 7: better RSVP representation

A little while ago I posted my first event, and soon after it came my own RSVP to that event. Back then I wrote a bit of text, because an Indieweb RSVP is a h-entry with a u-in-reply-to and a p-rsvp property. Since it’s a reply, I wrote reply text.

Most people don’t display the text for RSVP posts though, myself included. And today I RSVP’ed on a Meetup.com event, so after the ‘hey you need to post this on your own site first’ bells went off I made a new RSVP on my site. I forgot the text, so it looked like this:

It’s a bit yucky. So since I already have icons here for displaying the different RSVP values (yes, no, maybe and interested), I thought let’s re-use them.

I changed a lot of code behind the scenes too, because my h-entry template became a mess with all the different options it has. (But behind the scenes doesn’t count for #100daysofindieweb!) I now display RSVP’s as what I now call ‘shortposts’:

I also added my bookmarks to the feed at the homepage, because I like to show off my different types of shortposts. (They first where only available at /bookmarks, and I'm actually subscribed to that feed myself, so I can review my own bookmarks in my reader. Nice lifehack.)

Day 6: getting old posts back (part 2)

I got my posts back! :D

Unfortunately the quality of the dump isn't very good. Markdown is sometimes not interpreted, there are double posts (Tweets linking to my blog or Instagram) and a lot of images are at width:500px, which was my default width for over 10 years.

I've been coding some kind of post-match code, too see if I can detect the double posts. It quite hard, because a lot of tweets that are not the same post, are send within 15 minutes of each other, so published is not a good predictor. Trying to calculate the text similarity with levenshtein() seemed like the solution to all my problems, but there are quite a few mismatches still. I probably can't do this fully automated.

I now really have to go to do some real-life stuff, so I guess today just has to be the day I imported the posts and added the images. It's not that I haven't put in the hours today, it's just that it's way too much. (Or I need to be more efficient) You can check things out by scrolling way down in my feed, or at /blog or /tekstbeelden.

Day 5: getting old posts back (part 1)

I have good news and bad news. The good news it that I made a lot of progress in retrieving my old blogposts. The bad news is that I don’t think I can finish this today anymore.

tl;dr: I have my posts in Kirby-formatted .txt-files now, but I need more time to think about moving the images.

In an attempt to own my data, I posted all my Tweets and Instagram-photo’s onto my weblog. To do so, I started fresh, but I never got around to post the old posts back. I have 8000 posts on this site, but the old one’s are all social media reposts, no originals, even though I have had this domain since 2006.

There are roughly four periods between 2006 and now. In the first 5 years, I used a custom made CMS, written by a 16-year-old who taught himself PHP by reading a 175 paged book. (That’s me.) In 2009, I found it too insecure to keep it running that way, so I moved to Wordpress. Looking back at it it wasn’t so insecure at all. Yes, I was 16 years old when I wrote it, but I knew about SQL-injection at the time, and Wordpress isn’t known for it’s security. My own blog at least had security by obscurity.

At the beginning of the Wordpress era I made a fresh start. Luckily I found the old server still running, so my posts should still be there in that database. (You can send a GET request with Host: seblog.nl to my mother’s website to get the last pre-Kirby version.) If not, I have a PDF somewhere, but that’s not great for importing.

After a while I moved away from Wordpress in favor of .txt-files. I did a proper import back then, so that’s where I’m working with now: the .txt-version of my blog.

And those files are great, but hideous. Somehow I thought it was a nice idea to link them together with Javascript, so this version of my blog is not archived by the Web Archive. It worked like an infinite scroll: I had one file called ‘blog.txt’, which only contained a link to the last post. At the end of that post was a link to the next post. I think my 404-page showed a C implementation of the linked list, because that was my inspiration.

Next to ‘blog.txt’ I also had ‘verhalen.txt’ and ‘tekstbeelden.txt’ for other streams of posts. In my import I wrote a nice little recursion to get a list of all the stories, so I can tag them properly:

function findnext($from, $links, &$a) {
  if($links[$from]) {
    $a[] = $links[$from];
    findnext($links[$from], $links, $a);
  }
}

$verhalen = [];
findnext('verhalen', $links, $verhalen);

There is also a lot of Markdown-that-isn’t-Markdown in my posts, which complicates things. Oh, and Wordpress-HTML. I’m trying to regex a lot out of it.

The last problem I have now are images. In Wordpress and my own creation I stored all the images in one folder, but in Kirby the convention is to store them in the folder of the page. (Kirby is one big folder structure. I like that.) I need a way to either upload them when I post the posts via Micropub, or a way to identify the location of the newly created posts from the old slug, so I can move the files on the server. I don’t want to do it by hand.


This blogposts gets hopelessly long. Maybe I could’ve fixed my images-problem by now. But I want a clear head before I start importing. And a backup, but with 8000 posts that takes some time. I’ll resume tomorrow. At least I made progress!

Day 4: scheduled posts

I'm very late today. It's actually day 5 now in The Netherlands, but since I haven't slept yet I'm still calling this day 4.

Because I started so late, I chose an easy one for today. Unfortunately the easy ones always take longer than you expect. But hey, that makes this episode of #100daysofindieweb an episode!


I am now able to schedule posts on my blog. That's a simple one, because all I have to do is check wether the 'published' field of a post is in the past before I display it. It took a little longer because I had to do it in two places, the feed and the entry page.

The feed now only grabs posts where the 'published' is smaller than the current date, and my router now gives a 404 Not found if a post has a 'published' that is bigger than the current date.

Because my webmentions trigger on the first visit of the page, and the page never gets called, I effectively schedule webmentions with this too. Only one problem: I need to wait for a visitor to open the post once it's published to trigger the mentions. This includes the mention to Brid.gy/publish, to syndicate a post to Twitter. I leave that problem to another day though.

Anyway, it's a simple feature, but a neat one, so I'm happy with it.

@bitsoffreedom Ik heb al bij twee evenementen de klantenservice gemaild omdat hun ticketsysteem me verplicht om geboortedatum en geslacht in te vullen, vaak omdat ze hun ticketverkoop hebben uitbesteed. Hebben jullie daar toevallig een blogpost of pagina voor waar naar kan linken?

Day 3: relative urls

Today I started with fiddling with reply-contexts, but that turned out to be a giant mess, because I store my own posts in a different format than posts by others, and I wanted to re-use my template. I need to come back to that another time.


Yesterday, I mentioned Martijn in my post. And when I mention urls in my posts, my server will try to notify the other site that I mentioned that page. So yesterday my site notified Martijn’s site. At least, it tried.

Martijn is known for giving parsers a hard time. My site came up with http://vanderven.se/mention.php as his webmention endpoint, which is not correct. The following things went wrong:

  • I mentioned http://vanderven.se/martijn, which is not his canonical url. That’s http://vanderven.se/martijn/, ending in a /.
  • Kirby Toolkit’s remote::get(), which I used to fetch the page, did follow the redirect, but my script didn’t update the mentioned url accordingly. Since Martijns endpoint link is relative (webmention.php), that / does matter: it defines Martijns page as a folder, so his endpoint is at http://vanderven.se/martijn/endpoint.php
  • Finally, it turned out that Kirby Toolkit’s url::makeAbsolute($path, $home) did not handle relative urls well. It just intelligently added $path to $home, but didn’t actually solve relative urls where $home is a folder or file.

Martijn encouraged me to write some tests for Kirby Toolkit, because they use phpunit and all that stuff. It was nice, because I didn’t have any experience with this kind of testing (I just refresh the page and hope it works). Learning to test things is a goal for 2017 for me.

When I had a list of good tests, I started to change the function. I might be a little too refresh-frenzy still, but in the end I managed to pass all my own tests. After the pull request and with a little tweaking of my own scripts, I finally got Martijn’s endpoint:

http://vanderven.se/martijn/endpoint.php

So, Martijn, here is another try, still without your canonical /, just to see if it works now. :)

Ik weet het niet hoor jongens. Ik heb zojuist een stukje van 418 woorden geplaatst onder de categorie '100dagen500woorden', maar ik vind het gewoon gênant. Ik geef jullie de eerste zin:

Ik weet niet, maar op een of andere manier is een codehoofd gewoon een heel ander soort hoofd dan een schrijfhoofd.

Het is zo'n typisch stukje over schrijven. Daar schrijf ik er heel veel van en die probeer ik zo min mogelijk te publiceren. De bedoeling van deze challenge was dat er wat meer creatiefs naar boven kwam. Maar goed, bij proberen hoort falen. En daarna opnieuw proberen.

Dus denk dat ik dit deel van de 100 dagen ga omgooien. Het andere deel gaat prima en zie ik 'makkelijk' tot de 100 komen, maar dit deel voelt verkeerd. Naast '500 woorden' heb ik toch ook een soort kwaliteitskeurmerk voor ogen en dat kwaliteitskeurmerk is ten eerste niet reëel voor 500 woorden, 100 dagen lang, naast minimaal 5 uur zakelijk websites bouwen én je eigen weblog verbeteren, en ten tweede eigenlijk ook volstrekt irrelevant.

Mijn productiefste tijd ooit was juist een tijd waarbij kwaliteit 'och' was en waarbij het er om ging dat je er was en dat je dingen maakte. (Ik refereer hier aan het legendarische Waai.) Daarom ga ik dat gewoon schaamteloos opnieuw proberen: #100dagenwaaien.

Er volg zometeen een haiku om te beginnen met dag 2. (Want zo gaan die dingen.)

Meer laden