Reacties

Often it does not do everything in one job. It’s split it up into a smaller jobs. For example, it insert 100 products each time, instead of 10000 in one go.

The plugin does not split up jobs at the moment, the worker will just run for all 10.000 jobs, but if you're using the Cron setup, it will do it behind the scenes. (It will still take a while, but at least people are not waiting for it, and the work has to be done anyway, or else you shouldn't queue it in the first place.)

Instead of doing what I suggested with starterhook or the image hack, it could probably be solved with an ajax call.

Sure, that's the same thing. The important part is that you cut the connection to the client as soon as possible, and ignore that connection loss. You just want to trigger a work script, you are not loading something for the user. What if your job takes one minute, the user closes the page after 30 seconds, and your script terminates mid-work? If you're using an image or an AJAX call: return something, disconnect, then work.

Edit: note that the cron job is always preferred above the AJAX or img, because if you don't have visitors, you're job will not be done with the AJAX or img route.

For anyone who's reading along: if you just want to define a job and do it / trigger it somewhere else in your code, you can use Kirby's triggers:

kirby()->hook('my.action', function($name, $message) {
  echo 'hi ' . $name . ', ' . $message;
});

// and elsewhere:

kirby()->trigger('my.action', ['Seb', 'how are you?']);

Or take a look at Kirby 3's new Events:

use Kirby\Events\Events;

// the class with the events trait
class MyClass {
    use Events;
}

// create the event
MyClass::on('say', function($message) {
    echo $message;
});

// trigger the event
MyClass::trigger('say', 'hello');

Ahh, I see! Yeah, I am on shared hosting, but still have Cron (in my DirectAdmin panel, under advanced, but it's there), and most of the shared host I have seen here in The Netherlands have it enabled too. So that's why I took this cronjob approach first.

I totally agree the image looks hacky, but your starthook runs on pageload, right? Then you're just moving the work to the next request, which is even worse than doing it inline. (A person sending a webmention knows what it's waiting for, but the person who visits the homepage right after a webmention was requested just thinks the site is slow.)

The whole point about a queue is to get work out of the pageload, into a separate process. Any solution that triggers with a panel hook, within a plugin, etc, will defeat the purpose of the queue.

I hope so too! My testing works fine, but I haven't completed the Webmention plugin, so I am not using it live yet :)

I don't really understand what you mean by 'a plugin for cronjobs'. What you describe is more or less what this plugin does, or am I missing something?

Note that the 'X jobs waiting' button in the widget links to /panel/queue/work at the moment, which will try and work all the jobs from the queue. Probably not the best, but it's triggering (all the) jobs from the panel, kind of as you described.

I was also thinking about adding an option for a web-bug, to use for people without Cron on their server. That would be an image, like <img src="/worker.php">, and then in worker.php, send a one-pixel image, drop connection, and then do the work, described here.

And then there is a style of worker I can't use, but what is really a worker.php: a worker that you just run with $ php worker.php in your terminal, which has something like this:

// load all of Kirby once!

while (true) {
    while (queue::hasJobs()) {
        queue::work();
    }
    sleep(10);
}
// And then just keep checking for work, every 10 seconds, 
// never ending the php script.

Oh, and it might be nice to schedule jobs for the future. In my case, "when I posted a post... check back in 12 minutes and see if my mobile phone sent my location from that time to the server". (I have an app that tracks that, and submits it every 10 minutes.) So that's a function I might add.

Enough to improve! (But I'll be working on Webmentions first.)

Sad to hear, especially since you tried to use my plugins. But I will not try to convince you to come back, I know my plugins are lacking. They are just pieces of my own site I open sourced, but the way I cut them off might break things.

I am actively working on two new ones at the moment, with some 'official' IndieWeb libraries, with unit testing, hopefully even with panel widgets. They should be much more stable and user friendly than the old ones.

So I am not asking to come back now, but please, let me ping you when I got the new ones ready. I hope you will like those.

We're a long way from general adoption, but we can make steps.

I just wanted to add my 2 cents to this :)

I run my personal blog on Kirby, but since I've gone Indieweb, I've imported all Twitter and Instagram posts I ever posted. Recently I started to use Foursquare/Swarm, so I'm adding about 6 posts a day. At the moment of writing I have 8884 posts, in a folder structure with one folder per year, with folders per day of the year, with folders for every post (YYYY/DDD/N, I'm rewriting URLs to get /YYYY/MM/DD/N/optional-slug). On top of that I have views for only blogposts, photo's, checkins, and every category/hashtag is searchable.

I have to admit that I got rid of the panel, but that is because I don't like that it leaves empty fields written out in my content files. (I have a lot of optional fields.) But I do think my site could run with the panel. It might need one panel.page.create hook, but that's all.

I use Kirby's standard folder structure, but index it with a database. The database is just an index, so I can get rid of it at any time I feel like and regenerate it. The main content is stored in the .txt-file and if you visit a post-permalink, I don't access the database at all. The database is only used for the feeds and the category/tag-searches.

Because of all my optional fields, I have quite a lot of template-logic, so I cache the HTML-files of the 'entry' snippet. (one permalink version and one feed version) The rest of the page is still rendered without cache, it's just the entry part (the white block in my current design), and it's still fast enough to do the caching on the first load after update. I use this caching moment to update the database/index.

So I do the following on page-load:

• if there is a .html version, and the f::modified() of the .txt version is older, and the snippet is also not modified
  • return the cached html (stop here)
• if the is no .html version or the .txt was modified
  • update the index/db
• generate a new .html version from the snippet and return that.

This way I can update via Micropub, via the Panel, via $page->update() or even via FTP, and the db will be updated on next visit of that page.

The only caveat is that the post will not show up in the stream if no-one has visited it yet, so I have added an indexer::add($page) to my Micropub endpoint, and would have to add one on the panel.page.create (and why not also update) hooks. When updating via FTP I have to make sure to visit the page.

All in all, now my site is fast, and still has the benefits of Kirby's very readable file system.

Please shoot criticism my way now ;)