Minor thing, but... I feel lighter these days, so I wanted my site to reflect that. I always thought of <a href="/category/eee">#eee</a> as the lightest of grays, because there is only <a href="/category/fff">#fff</a> (white) as the next step. The six-digit notation was too granular for me. But it turns out that, for a background color, <a href="/category/f6f6f6">#f6f6f6</a> actually gives a whole different feeling. It's so much clearer!
Another thing you might have noticed, it that I broke the streak of consecutive #100daysofindieweb. This has to do with the lightness, I guess. I came home after a nice evening of talking about writing, and I just didn't want to get into the coding state of mind again. So I decided to leave it that night.
The returning problem with 100daysofindieweb, for me, is that I haven't figured out all the things I want from it yet. In the last entry, for example, I used Aaronpk's emoji library to display reacji in a certain way. While this is a nice thingy, there is an underlying question that I did not ask myself: do I need this?
For me, it was a recurring theme, remembering to do something IndieWeb around 0:30 at night, and just inventing 'problems' to solve, preferably small problems with quick solutions. I implemented a lot of stuff I think is useful, but also a lot of things just because other people did it before me.
A few days ago, I noticed that Whatsapp added the 'photo stories' feature, invented by Snapchat and copied by Instagram. My first reaction was: hm, how can I do this on my own site? While that is a nice question to ask oneself, it is useful to also ask the question wether you actually need to have such a thing on your site. Photo stories are hard to copy, because you can only add a new photo to them if you take that photo directly from the app. Also, the discoverability and way of viewing them is 80% of the feature. Their temporary nature also makes them less suited for the web. All in all, I shouldn't rush into this, and post photo's that disappear after 24 hours, just because I have a day to fill.
I will continue the 100 days. I will just sometimes take some time in between days, to think about the route I want to follow. To ask myself the question: do I need this? Minor tweaks make big changes over time, and it's okay to redo some stuff you did before. That's the way your understanding grows. But no more big new things for me now. I more or less have what I need, I just need to improve some things.
A few days ago, Aaron Parecki made an emoji detector for PHP. I felt I had to reply to that in style by using it.
So, now my site supports reacji, a reply which contains only of one emoji. In theory, it did already support this, of course, but now I show them as a 'shortpost', next to likes and other things I use icons for:
Today, I replied to some people on Twitter, and I was surprised to see 'twitter.com' turn up on my site as the author of the post I was replying to. I now have XRay with Twitter vision, right? Why didn't it work?
It turned out I only use XRay for likes, bookmarks and reposts, but this reply context relied on an old piece of code I wrote myself. It fetches a h-card for the URL, which works for IndieWeb sites with Microformats, but not for Twitter posts.
So I fixed that: now XRay fetches context for replies too:
This is the first step in actually displaying the full reply context on the permalink page. The data is there now!
Today is a (almost) no code day. I wrote a privacy policy.
Now that I have ways to log in to my site, I kind of need a page, because I am handling with personal information. I believe it's even required for the Twitter login thing, so here it is.
It feels more formal than I want it to be, so I tried to keep it informal. I also translated the Dutch version into English, because I have embarked on this multilingual thing now with this 100days series and English visitors need a privacy policy too. The document just comes down to explaining what I do, and asking to contact me if one disagrees. I don't do crazy stuff, so I think it's fine.
The bit of 'code' I wrote is in the footer, where there's now a link to the privacy policy.
At some point while building the foundation for this site, I decided to save all dates in UTC. I like UTC, I like timezones in general, and here in Europe we're not that much off UTC either.
In about a month from now, though, the gab between UTC and CET will grow again with one hour, due to 'zomertijd'. And even though there was an explicit 'UTC' behind all the times, it was still confusing for Dutch visitors, who where expecting 'normal' times.
I now show times in Central European Time. There's an explicit 'CET' behind all the times. The Microformats times are still on UTC though, but I added an explicit +0000 behind all the timestamps, to make clear that it's UTC.
One of the things I keep changing my mind about, is reposts. They are totally normal on Twitter, but seem to be totally weird on your own site. When signing up for Twitter, you give permission to Twitter to let other people retweet your tweets. When signing up for IndieWeb, yeah, non of that exists.
Still, I reposted a post by Loran Kloeze two weeks ago. It was marked up with h-entry, so all I had to do was just feed my Micropub endpoint the URL, and my site automatically fetched the post for me. At that time, I thought it was good to do so, because the post was important and chances were Lorans server would not have handled it if it went viral. I mirrored it, so others would be able to mirror it. Sounds good, right?
I still did not ask permission for the repost to Loran. (And I doubt he knows about it... maybe he gets a Google Alert after a while? He does not support webmention.) Yet the whole post was available at my site. What about those copyrights? He has removed his Google Ads now, but at that time they where there, and I was potentially keeping visitors away by giving them an ad-free environment to read the post.
So, today I changed some things about how I repost. I only show the first 50 words in the stream, like this:
There is a link to read more at the original URL. Tweets are, with 140 chars, mostly kept as is. I am just quoting, but the whole post is so short that I display all of it. The longer posts are cut off, to encourage people to read it at the source. If the permalink expires at some point, I can always decide to show the whole post again. Or maybe I should delete the repost then? That depends on the post.
When visiting a permalink of a repost on my site, you will get redirected to the original post. This way, reposts only appear on my stream, and I don't host their content under a URL on my site.
Okay, I did not implement receiving private webmentions today. Reason for that is that I also need to do some other things in life, but on top of that: I don't like the way I am storing webmentions right now. I feel like I have to solve some other things about that, before I can actually implement receiving private webmentions. I could just go for it, but then I am just adding more on top of this thing I don't like, which makes it harder to change, etc. I will make receiving private mentions a gradual thing, spread out over more days.
I still need to do something today, but it was hard to come up with anything, for everything seems to big for the time of the evening. So I decided on something minor: I now randomise the post that are visible in the 'Read' view of Lees. Wait, let me first introduce you to that.
Lees, the Dutch word for 'read', is my IndieWeb reader. It is all quite experimental and unstable, so I don't recommend using it, although the code is open. (Do I have a good license there? No. I need to look into that.)
Lees just polls different websites via a cron job and pulls Microformats from the posts. It presents those posts in a 'new' feed, 10 at a time, oldest first, so you don't miss a thing. Then there is a button that says 'mark all as read', which marks all the posts in the current view as read. This is great for reading tweets in bulk.
Some posts need more time, though, so there is a 'read later' checkbox. Checking it will, after you click 'mark all as read', not mark the post as read, but send it to 'read later'. This way you are just glancing at the 'new' feed, and picking the things you really want to read.
As you can see above, you can also like, reply and bookmark directly from Lees. Well, reply is actually just a link to Quill, but it's still works. There is build-in Micropub support for the on-click-post ones.
My problem is that Lees does not yet filter stuff. I follow too much people on Twitter, so now I actually removed Twitter from Lees and check the Twitter-app again. I also have a list of articles I would like to read in my 'Read' stream, but I see only the same ones over and over again, the oldest 10. Every time I see them, I think 'yeah, I should read those', and that guild does not really convince me to do so.
So in an attempt to get myself back to Lees, I randomised the posts in the Read section. Their order is now shuffled, so I see 'new' ones when I login. Those are all posts I handpicked for looking interesting. I hope I am now more motivated to actually click some of them, as I rediscover them. Also: if I now keep seeing a particular article that I don't read... I can remove it with more confidence.
So there it is, day 31. Writing this blogpost took longer than the thing, but I'm happy with the result, that's what counts!
Now that I support both private posts and webmention, it is time to put those two together and send private webmentions. In short, this is a way to notify URLs you mention in your posts, while still maintaining the privacy of the post.
Following the description on Indieweb.org, my private posts now send a HTTP Link header to my new token endpoint. When I send a private mention, I also send a code: a JWT token with the target and source URLs of the webmention and an expiration of 2 minutes. The other side can discover my token endpoint by looking at the Link header, and send the code in a POST request.
The code is then exchanged for an access token, but now with a me, which is set to the URL of the mentioned post, a page, which is my page, and a nonce to mess things up a little. This access token expires in an hour, which is plenty for automated fetching, but might be a bit short for people verifying manual. Normal people can still try and login via IndieAuth though!
However, the mentioned URLs and the whitelisted people are different fields, so they can be different. In fact, they should different: if I mention someone.com/a-post, I might to give someone.com access to the post by logging in, but only if the owner of someone.com/a-post is actually someone.com. In case of twitter.com/url-to-tweet, I don’t want to give twitter.com access. I’m not sure how to solve this yet, so I will do it manual until it hurts.
Also note that I only implemented sending private webmentions today. Please don’t private-mention me now. I will do that part tomorrow.
Now for the Special Valentines Edition™: I just created a post with a u-like-of pointing to a personal site / profile of someone I like. The post is of course private and only I and that person can see it when we log in. I have sent a private webmention for it. The only sad thing is that the other side did not support private webmention and nothing got sent, so I guess it does not really count as my first private mention. Tragic Valentines Day for me so far.
Yesterday, I found a weakness in Quill, so I notified Aaron and he fixed it today. (Actually, I discovered it today at 1:30, and he fixed it yesterday at 20:30. Timezones are magic!)
On Day 15 of my 100days, I found a vulnerability on my own site. I promised to blog about it, but I actually waited, to give Bastian Allgeier the opportunity to fix his site too. Then I just postponed the writing some more, because life happens. I finally wrote about it today.
This brings my count of responsible disclosures to 2, and I’m a bit proud of that.
Two weeks ago I wrote that I hacked my own site. I think it’s important to share how I did it, to make people more aware of possible vulnerabilities, so they can find them too. If others didn’t write about their findings, I wouldn’t have found this one.
I did my best to reach out to people using the same code. If you are using the Kirby Webmentions plugin or my fork of it, please make sure to update!
Webmentions
As some of you may know, my site supports webmentions. In short, this enables me to show replies underneath my posts, that are written by people on their own site. If you write a reply, link to me, mark it up with Microformats and send a webmention, my site fetches your post and shows it as a reply. I use a service called Bridgy to also receive comments from Twitter and Instagram. All of this is automated and very cool.
However, while very cool, it is also potentially dangerous to show external content on your site. The vulnerability I found is an example of what can go wrong.
If you look around on my site, you see I do not only show the content of the reply, but also a picture of the author, if provided. This is especially nice when showing likes:
In order to protect visitors of my site from other security issues, the plugin downloads the images and shows those downloaded ones. This way my visitors only deal with my server, and not with the servers of everyone who liked my post. It’s a nice service, but it also means that I move the problem: I now have to handle those images with care on my side. My server takes the fall for my visitors.
The problem is: my server just downloads whatever image you give it. In most cases, this will be a nice avatar I can display for my friendly visitors. But one can think of a case where a not-so-friendly visitor feeds my site something else than an image. The plugin of course checks if it’s an image and rejects files that are not a image, but it’s still worth a try.
So, what did you feed it?
Since my server runs on PHP, the nicest thing for an attacker to feed my server is a PHP-file. That way, you can run whatever code you want on my server, doing all kinds of evil things. However, just straight off feeding my site a PHP-file did not work. The plugin is not crazy. It checked wether the MIME of the file was an image of type jpeg, png or gif. It rejected an image.php file like this:
echo "hi!";
Using image.jpg as filename would fail too: the plugin saw that the file had no MIME of an image, so it did not download it. This was the point where I went to bed with a feeling of security: my site was safe and I could not get a php-script in.
The next day, however, I had second thoughts. I needed a real image for my new plan, so I took a screenshot of a smiley. I then opened it in notepad and added the following to the bottom of the file:
<?php mail(‘my.email@gmail.com’, ‘Seb’, ‘hi’);
I then renamed the file to image.php, because you need the PHP-extension in your file to let the server run your code. The last step was disabling PHP on my test-server, to prevent the test-server from executing the code and send mail me. The code just appeared at the end of the image.
I then made a test-post with a u-like-of set to the URL of a post on my blog, and a p-author h-card with an <img src="/photo.php">. It was a like, with an author and an my bad image.
And it worked.
The server sees the image and checks for the MIME, which was image/jpeg, because it was an image. It then downloaded it, including the un-executed PHP string in the bottom of it. It changed the name of the image into the SHA1-hash of the original image-url, but then it appended the extension of the original file, which was .php!
My server then had a file called a266d629bb26d74752080bb1b95bbd0a488bea53.php, which was linked as an image in my post. Every time I refreshed the page, the snippet of code in the bottom of the file got executed, so it sent an e-mail to me.
In this example, I sent an e-mail, but it could’ve been anything.
How to solve?
First off: check your input! And then check it again. A crucial thing for PHP-files is that they get executed if they have the .php extension, so you should not rely on user input for that. Change the filename and change the extension.
Bastian updated the plugin, so now it does not only check for MIME, but also only accepts files with the extensions jpg, jpeg, png and gif. Only if it has a correct extension, it downloads the file, and it checks MIME twice, both before and after the download. I think it’s locked down pretty well, although it still feels a bit scary.
Aaron Parecki, who did this way of showing likes first, uses an external service for his webmention images, and that’s not a bad idea either. If someone manages to get in something bad, it’s not on your the same server as your site. It could also be a good idea to turn off PHP for your upload folder, if you have that kind of access to your server.
Final words
I really like this webmention plugin! It’s thanks to this plugin that I know IndieWeb and all the wonderful things it brings.
But while the plugin and IndieWeb are nice, it’s also good to keep and eye on security. At this moment, webmention is relatively safe because not many people know about it or use it. Although it can be a lot of fun to have a post of a friend automatically show up beneath your post, we have to be aware of the risks of showing content of external parties.
Taking yesterdays private post adventure one step further: it’s now also possible to log in to Seblog with Twitter. I still recommend having your own site and use IndieAuth, but I have to be pragmatic: a lot of my friends do not have their own site. (If you want your own site, like this, just give a shout, I can help.)
To make this login adventure a bit more visible, I also added a ‘Log in’ link in the top right corner of every page.
Unfortunately, there is not much going on for people who logged in via Twitter. The private post of yesterday is still only visible for those who were on IRC-people yesterday. But it’s a step!
A few months ago I wrote that I made 'privéstukjes' on my site. The implementation was as simple as one field called 'private' that I would give a value (true for example). If the field has a value, my site returns a 401 Unauthorized header and a page explaining that there is nothing to find there. I did not implement a way to log in, for I only used it for (two) drafts / pieces of bad writing I wanted to keep for myself.
But what is the fun of a private post when nobody can see it?
Today, I implemented a login for my private posts. It was a bit more work than I thought, getting IndieAuth to work and thinking out all the different states of a post, but I made it.
Posts can now also have an 'audience' field, where I keep a comma separated list of URLs that have access to the post. I plan on expanding or replacing that with predefined lists (friends, family, etc.), but for now this works fine.
If one is not logged in, the private post page returns a 401 with a (Dutch) explanation of what is going on. There is also a field to put in your personal URL for an IndieAuth login. If you login, but are not on whitelisted to view the post, you will get a 403 Forbidden, and a Dutch explanation. (Translating my site based on visitor language is still a to do.) If you are on the list, you will see the post.
I have made a test post, so you can try out for yourself. All URLs on IndieWeb's IRC-people page, at the time of writing this, are whitelisted. Have a try at it if you're on that list!
Oh no! I almost forgot to work on something IndieWeb today. This was partially the plan, because I also want to focus on some writing (which did not really worked out), but I still wanted to do something small in the evening to keep this thing going. And that part I almost forgot.
So here is a really small change: Tantek was talking about reply contexts for RSVPs, and I said I only show the name of the event, and asked if that counts. And yeah, in a way it counts.
I wanted to add a bit more though, so now I also show the date of the event:
A very minor thing, but hey, it's a thing! I don't really want to add more details to my RSVPs, because I like them small and people are one click away of knowing more about the event anyway.
A blast from the past: I have an RSS feed and it's available at seblog.nl/feed.rss. (I also have seblog.nl/feed, which is actually the h-feed version of it.) I made the RSS feed a while ago for Martijn, who I believe is my only subscriber at the moment.
Back when I made the feed, I didn't want to put in much work for it, because I already have a couple of rich h-feeds. So I made it so that "if a post has a title, give that, else, just call it 'post'" and "if a post has content, give that, else, just say 'A post on Seblog'". Very lazy. It turns out I actually have a lot of posts that have neither a title nor content. Things like my likes, reads and bookmarks all have no title nor content, because it is stored in different fields. So my RSS feed ends up being filled with contentless posts.
I am still lazy about my RSS feed, but I improved it a bit. I now just call the same snippet that produces the HTML for the frontpage for the content of the feed item. This means the whole post is in the RSS feed. Probably not the best way to do it, but hey, who uses RSS nowadays anyway?
Today I was at Homebrew Website Club in Utrecht. We had some talks, showed some stuff, and then coded for a while. (You might say that we switched the quiet writing hour and the talks around, but we were not silent all the time.) Jesse made a first version of a blog section on his site and Martijn did some tweaking on his habit tracker.
I made it easy for myself: I just copy+pasted my existing [media endpoint](https://www.w3.org/TR/micropub/#media-endpoint) into my Indieweb Toolkit. Then I spent the rest of the time tweaking, cleaning and improving my code. The whole endpoint is now contained in a single static method, which takes a string with the path to the upload folder, and a string with the public URL of that same folder. It validates access tokens, saves media with a sanitised filename and an unguessable prefix, like vx5dgk-image.jpg, and then returns the URL of the uploaded file in a Location header.
Okay, I admit: I already did this yesterday, as you can see from my previous retweet. But I wanted one day off. One day without screens. So that’s today.
Earlier on I made my site like tweets when I post a like to a tweet on my site. This means I get just feed my Micropub endpoint an URL as a like-of, and don’t think about it anymore. Now I added reposts to this too, so I can just paste a link in Quill and it’s ok.
I also seemed to miss pictures in reposts since I redid them, so I fixed that as well. It’s the little things. See you tomorrow!
In December I already posted some plain text notes tracking my reading progress in Stoner by John Williams. (The Dutch translation, by the way, because I got that one somewhere in a very portable version and I was on a city trip to Vilnius.) I already planned to upgrade those plain text notes to a more standarized format, so that I would have /read posts.
Before I could add them, though, I needed a way to give them URLs. So now I have this public list of books I own / owned / want to track, on my new library page: /bieb. All books in there are marked up as h-cites and have a url in the format https://seblog.nl/isbn/9780123456789. Their u-uid is set to isbn:9780123456789, because that seems to be a valid url scheme (but browsers don't know what to do with it).
The next step was simple: I just expanded my shortpost type with a 'read' type, that looks for a read-of field. Thanks to XRay and my existing code, the seblog.nl/isbn/-url gets expanded to a title and author. I also added a page field, because I posted when I was at certain points at the book, and I actually haven't finished it yet.
Today I added a very basic Micropub endpoint to my Indieweb Toolkit. A Micropub endpoint is a hard thing to keep generic, because every site has a different way of storing data. The idea about the endpoint in the toolkit is that it does all the things that are the same about Micropub.
So here's an example of what it can do. This one simply writes a YAML file with the received fields to a folder. Please note that it overwrites any post with an existing slug. It's only an example.
The idea is that you can put this code where-ever you want. It can be at /micropub.php or in some route you define. The new endpoint::micropub() does all the authentication for you and returns the appropriate header. All you need to do is pass in a 'create' callback function, which takes the fields and returns a URL. endpoint::micropub() will do the redirect with a 201 header.
At this moment it only supports 'create' with x-www-form-urlencoded, but JSON and 'delete', 'undelete' and 'update' will follow. I also want callbacks for different mp-synticate-tos, and of course there is the ?q=config query. But I like this idea, and will use this idea to clean out the code of my own endpoint.
Last caveat: at this moment I have not implemented the IndieAuth class in my toolkit. You can steal the class from my kirby-micropub plugin, but I want to clean it first before putting it in the toolkit.
Today I drafted a SiloAPI class in indieweb-toolkit. The idea is that it gives an easy way to interact with different Silo API's. In some ways this is a bad idea, because all API's work different and have different capabilities, but I like the idea, even if it's bad.
One API call that is available for both Twitter, Instagram and Facebook is to create a like. So that's where I focussed on first. Today, I made SiloAPI a wrapper for the TwitterOAuth PHP library, and it now supports ::like() just like micropub::like(). You still need to pass in the name of the silo though, but maybe I'll detecting the silo from the URL later.
Then I realised that Brid.gy probably can do this for me too, since it already syndicates my Tweets for me when I ask for it. So I just made my Micropub endpoint call Brid.gy publish for every like-of that is a twitter-URL.
I will add to SiloAPI though, because I want to autolike to Instagram too!
Today is a weird day. Fixed a lot of things, but also did kinda useless things. (I learned more about HTTP requests by trying to create a Micropub request using netcat, and at some point, I was chatting with myself in two Terminal windows with nc -l localhost 8080 in one and nc localhost 8080 in the other, completely by accident.)
Then it was 1:30 at night.
Earlier on in this series, I fixed my repost context. These contexts where based on a .json file I exported with my whole Twitter archive. A few days ago I made XRay look up likes for me. Today, I retweeted someone, and since .json files don't magically appear on your server (luckily), it didn't show any repost context.
So I combined the two and now XRay fetches the repost context for me. I don't use the .json file anymore, but keep the data the same way I keep likes: in a refs field.
Today I extended my new Indieweb Toolkit with a Micropub helper class. This allows you to easily send Micropub requests from your PHP code. Here’s an example:
// Set URL and access token of the endpoint to use
micropub::setEndpoint('http://yoursite.com/micropub', 'xxx');
micropub::reply('http://example.com/a-nice-post', "Oh what a post!");
micropub::like('http://example.com/another-post');
micropub::rsvp('http://example.com/an-event', 'maybe');
$newURL = micropub::post([
'name' => 'Custom posts are possible!',
'content' => 'This is a story about (...)',
'category' => ['story', 'custom'],
'mp-slug' => 'custom-posts-are-possible',
'mp-syndicate-to' => 'https://twitter.com/example',
]);
go($newURL);
Note that this only creates posts, and relies havily on the server you are making requests to. The mp-syndicate-to in the above example indicates that the remote server needs to post the entry to Twitter also, this helper doesn’t do a thing for that.
A drop-in Micropub endpoint is hard, because every site stores it’s data different, but I intend to extend this toolkit so that the different parts involved will be easier. (De facto making my own code more re-usable.)
Twitter
Instagram
LinkedIn
Github
Strava
Facebook
