Log in
Seblog.nl

#howto

Adding Micropub to your IRC bot

In last weeks IndieWeb newsletter I read a blogpost by Sven about having notifications for webmentions and such via IRC. He described more or less the setup I have too: using Aaron’s TikTokBot framework, running on the same machine as a private IRC server for it and your IRC bouncer. In my case, the machine is a Raspberry Pi in my living room, but Sven’s guide describes more or less what I did too, so please check that out.

I divided my webmention notifications into webmentions and silomentions (that’s backfeed from services like Bridgy and OwnYourSwarm) and I also get notifications for logins and Micropub requests, so I can keep track of what’s happening on my blog. But that’s just expanding the notifications. I added some other functionality I wanted to talk about.

Hey bot, please like this post

The one thing that made me post on my site the most, was adding Micropub support. There are various Micropub clients out there that you can use to just write a blogpost, to post a like, to import checkins from Swarm or what have you. But since it’s so simple (once you know how), I also made a lot of Micropub clients out of Workflow, Paw or, in this case, my IRC bot called Bop.

I can now say like https://example.com/a-post in the same channel my notifications come in, and Bop will post a like-post on my site. My site will then post the webmention and if the other site accepts it, it will be shown as a like under the post, just like that.


Please refer to Sven’s blogpost to learn how to set the bot up. I also assume you already have a site that supports Micropub. I will go into obtaining an access token a bit, but that might be a tricky part and depends on how your Micropub endpoint works.

Getting your bot hooked

Sven mentions the file ‘hooks.yml’, which contains hooks: [] for him. The hooks-file defines the things the bot will respond to, and since Sven only want to receive notifications, not talk to his bot, he does not need hooks. But we do. Replace the contents of the hooks file with the following:

hooks:
- match: "^(like|bookmark|rsvp|yes|no|maybe|interested) (https?\\:\\/\\/[a-zA-Z0-9\\-\\.]+\\.[a-zA-Z]{2,}(\\/[^>,\\\")\\s]*)?)"
  url: "http://localhost:8000/micropub.php"
  channels:
  - "@seblog"
- match: "^(tweet|note|fb) (.*)"
  url: "http://localhost:8000/micropub.php"
  channels:
  - "@seblog"

As you can see, I like to match for a URL to support like, bookmark and rsvp, and for anything to support notes on your own site, Twitter and Facebook. Feel free to change the syntax to your own needs.

Under channels, I defined @seblog, which means ‘every channel in the server seblog’. That name comes from your config.yml, and it’s a safety measure, so that when I connect my bot to Freenode also, it won’t accept likes from other people. I am the only person connected to the private IRC network Seblog, so that will be fine.

Creating the Micropub.php

As you saw, the hooks define a URL that will be called by the bot. I specified that to be on localhost:8000 and point to micropub.php. In your TikTokBot folder (or where you want, really) create a folder called ‘server’ and a file within it called ‘micropub.php’. Let’s start with the following to test it out:

<?php
header('Content-Type: application/json');

echo json_encode([
  'content' => 'Hi, you called for a Micropub?'
]);

Then go to the newly created folder and run the command php -S localhost:8000. This starts up a webserver from that folder, so if you now go to your private channel and say note This is a test, you will see that the bot responds. If you say something that does not match the regex, the bot will stay silent.

Give me my data back

If we want to send data to our Micropub endpoint, we need to make sure that our bot has that data. Let’s make the bot echo whatever we said, so we know that he heard us.

TikTokBot gives us a JSON object of the message via the POST data. Long story short, you can get it by using the following:

<?php
header('Content-Type: application/json');

$message = json_decode(file_get_contents("php://input"));

$full   = $message->content;
$action = $message->match[0];
$param  = $message->match[1];

echo json_encode([
  'content' => "Hi, you said '$full', you wanted to $action $param?"
]);

Now, if you say like https://seblog.nl/2017/08/19/6/micropub-irc-bot, the bot will respond with Hi, you said 'like https://seblog.nl/2017/08/19/6/micropub-irc-bot', you wanted to like https://seblog.nl/2017/08/19/6/micropub-irc-bot?. Try it out, and different actions too, so you know it works!

Make sure not to echo the full message, because since the bot listens to itself, that message will trigger itself again, resulting in the bot repeating the message over and over again. (Just restart the bot if that happens.)

Sending a POST request

A Micropub request is, in the end, just a POST request. Here’s a little helper function to send one for you. Just start with this code at the top of your file, right after <?php.

function send_post($url, $fields, $bearer = null) {
  $ch = curl_init();
  curl_setopt($ch, CURLOPT_URL, $url);
  curl_setopt($ch, CURLOPT_CUSTOMREQUEST, 'POST');
  curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);

  if ($bearer) {
    curl_setopt($ch, CURLOPT_HTTPHEADER, ['Authorization: Bearer '.$bearer]);
  }

  $body = http_build_query($fields);
  curl_setopt($ch, CURLOPT_POST, 1);
  curl_setopt($ch, CURLOPT_POSTFIELDS, $body);

  $resp = curl_exec($ch);

  if(!$resp) {
    $resp = 'Error: '.curl_error($ch);
  }

  curl_close($ch);

  return $resp;
}

Next, I have a plain and dirty if-chain, to determine the action. You can add this right after you defined $action and $param:

if ($action == 'like') {
  $fields = [
    'like-of' => $param
  ];

} elseif ($action == 'bookmark') {
  $fields = [
    'bookmark-of' => $param
  ];
} else exit();

send_post('https://seblog.nl/micropub', $fields, 'xxx');

Make sure to replace that URL with your own Micropub endpoint, and xxx with your own access token. I’ll talk about obtaining one at the bottom of this post.

I don’t have the bot say anything after this, because the bot will give a notification once a Micropub request is made, which is enough feedback for me. You could also return whatever send_post() returns, I leave that up to you.

More post types

As you can see in the code above, you can just add fields to the array. Here are some other elseifs I use.

} elseif ($action == 'tweet') {
  if (strlen($param) > 140) {
    exit(json_encode(['content' => 'Text is '.strlen($param).' chars!']);
  }

  $fields = [
    'content' => $param,
    'syndicate-to' => 'https://brid.gy/publish/twitter'
  ];

} elseif ($action == 'fb') {
  $fields = [
    'content' => $param,
    'syndicate-to' => 'https://brid.gy/publish/facebook'
  ];

} elseif ($action == 'rsvp'
   or $action == 'yes'
   or $action == 'no'
   or $action == 'maybe'
   or $action == 'interested') {

  $fields = [
    'in-reply-to' => $param,
    'rsvp' => $action == 'rsvp' ? 'yes' : $action,
  ];

  if (strpos($param, 'facebook.com')) {
    $fields['syndicate-to'] = 'https://brid.gy/publish/facebook';
  }

} elseif ($action == 'note') {

  $fields = [
    'content' => $param
  ];

See how easy that is? Please make up new ones and tell me about it!


I’m particularly proud of the tweet one, which will, thanks to the exit(), not post the tweet unless it fits the 140-chars-rule.

Please note that the use of the syndicate-to fields (or the newer mp-syndicate-to) depend on your own website. This only gives my webserver the order to syndicate, it does not syndicate by itself. If your server does not know how to syndicate to Twitter, it will not work, but explaining how to do that is it’s own tutorial.

In real life, I actually have note set to 'private' => true and 'audience' => 'http://seblog.nl/, so it’s a private note. But: 1) I don’t expect many people to support private posts, 2) how many people are interested in posting things to their website only they themselves can see? and 3) I want to change the 'private' => true to 'visibility' => 'private', or something like that.

That’s it!

There you go, a customisable Micropub chatbot. Please double check that it’s only doing it’s thing on your private IRC server, and not on any public servers where your bot might lurk (if you use any). Have fun liking and posting!


POST-scriptum: Obtaining an access token

The question “what do I replace ‘xxx’ with?” is a hard one to answer, because it totally depends on how your site handles access tokens. Most Micropub clients ship with an IndieAuth flow to obtaining one, but since we’re making this one ourselves, we have to get one manually.

You can do this by following the steps on the wiki, if you have a way to send POST requests. You can also log in to a Micropub client like Quill, which shows you the access token it got, and use that. Since this all depends on your own site, it might offer an easier way, or it might not.


Update 2017-10-04: In an earlier version of this post, I did not set a Content-Type: application/json-header, so the bot didn't actually respond. Setting that header at the top of your file will fix that.

Switching to ZNC on my Raspberry Pi

I’ve been following the #indieweb channel on IRC for a while now, sometimes more active than other times, and my several methods. In the beginning, I just read the logs on chat.indieweb.org, but when I wanted to say something I had to login with Textual.

Back in Amsterdam I had a bad internet connection (they probably tried to block IRC) so I wasn’t using it a lot. Then there was the bridged Slack channel, which I used intil my first HWC, where Martijn introduced me to thelounge. I installed it on a Raspberry Pi I had lying around and used that for quite a while, until I became inactive and shut the thing down.

Unfortunately I haven’t been able to make it work again after that, so I was in need of something else. Meanwhile I got used to Textual again, and I kind of liked that, except that it disconnected every time my laptop went to sleep.

Ok, so, ZNC you said?

Yeah, enough history! Let’s get to it. ZNC is an IRC bouncer, which connects to IRC for you. When you connect to ZNC, ZNC will give you all the messages it received since you where away. So accedentally closing your laptop? No worries, just open it and you receive all messages again.

You still need some computer to be online all the time, but I had that same Raspberry Pi (first edition model B), and it’s perfect for this job.

I first followed the installation instructions here, first downloading the latest source tarball, and following the unpacking and configuring described on that page. Later on, I had to use a different flag on configure, so I added that one here for you. I used the following commands in order:

wget https://znc.in/releases/znc-1.6.5.tar.gz
tar -xzvf znc-1.6.5.tar.gz
cd znc-1.6.5
./configure --enable-python
make
sudo make install

This all takes a while, especially the make part.

I then made sure I opened a port on my router towards my RPi, so I could access it from outside my home-network, when I’m on the go (look for the NAT settings in your router).

Extra modules

Then for extra modules. The webadmin was on, so I could just go to my home-IP + my new port, let’s say https://192.0.2.0:8000. I had SSL enabled and bad certificates, so you might need to trick your browser into accepting them. In the global modules, I enabled chansaver, lastseen and log. I also had notify_connect, but it felt too noisy for me. Don’t forget to hit the ‘save’ button.

In ‘Your Settings’ I have chansaver and controlpanel on, I believe by default.

For other module configuration I connected to ZNC using Textual. When you add it, you can give it a connection name (this is for yourself). The server address is your home IP, and the port the outside port you put in your router. The server password is the one you put in ZNC in the ./configure step. But then comes the part where I was puzzled: go from the ‘General’ to the ‘Identity’ view, and add your Freenode IRC nickname as ‘Nickname’, but your ZNC username as Username. I used sebsel@Mac/freenode, to identify it later. The personal password is for the NickServ password on Freenode.


After connecting I did /msg *simple_away SetTimer 0, because I want to be set to away the moment all my devices are disconnected from ZNC.

Because I want to have Textual on my Mac, and this new-found Mutter on my iPhone, I have to get ZNC to manage multiple clients. By default, ZNC sees you have read your messages on your iPhone, so it does not send it to your Mac. To keep track of multiple clients, you need the ClientBuffer module.

Since I still had the source, I could just compile the module. So I did:

cd ~/znc-1.6.5/modules/
wget https://github.com/jpnurmi/znc-clientbuffer/blob/master/clientbuffer.cpp
cd ~/znc-1.6.5/
make
sudo make install

After that, I got back to Textual, and did /msg *status LoadMod clientbuffer. After that, you can /msg *clientbuffer help and send *clientbuffer some commands as messages. I did AddClient Mac and AddClient iPhone. You can do a ListClients too to see what you got.

Adding Mutter to the game

On the iPhone app Mutter, I added a network with again my home IP as the server, my nickname sebsel and under Advanced use the port I opened, added the password of ZNC in the first password field. The ‘username’ under Advanced gets the username for ZNC, so I made it sebsel@iPhone/freenode, to let Clientbuffer identify it as my iPhone. If you have notify_connect on you will see that *status notifies you about logging in with your iPhone.

Warning: I didn’t get this next part to work

Next I saw that there is an option to receive notifications from ZNC to your iPhone via Mutter. You will need their mutter-push module for that, and that requires ModPython, so that’s why I added the flag --enable-python during the first make.

As their explanation says, first grab the mutter package, then make sure you got the right python libs, and then move mutter.py to ~/.znc/modules/.

cd ~
wget https://bitbucket.org/jmclough/mutter-push/get/master.zip
unzip -d mutter -j master.zip
cd mutter
sudo apt-get install python3-pip
sudo pip3 install requests
cp mutter.py ~/.znc/modules/

In IRC, make sure to /znc loadmod modpython and then /znc loadmod mutter (or send a message to *status with loadmod modpython etc.).

This last part of getting notifications is still not working for me. I still wanted to post this though, because all the other things work. Hope I’ll get back to notifications soon, when I do get them to work.