Webmentions

A response to First Look at Webmentions

I have spent the last few days trying to fully support webmentions on this blog so figured I’d take the opportunity to write it up.

Webmentions are remarkably well named for a programming project. They provide a method to formally mention one part of the www from another. Implementing support for them came in 3 parts, sending, receiving, displaying. I began a few days ago, with the sending.

Sending Webmentions

You create a webpage that mentions another, then you tell the other page that you did it, that’s it.

To mention another page you need to decorate your html with specific classes (not all required) so that it can be parsed easily later. I’ll do my best to describe the ones I use here, but this article by Aaron Parecki, is likely a better resource for this.

  • h-entry - The top level element, likely a div or article element, that holds the content of the mention
    • p-name - Title, if applicable, of the mention (e.g. if a blog, it’s H1)
    • dt-published - Element with the date the mention is published.
    • u-in-reply-to - On an anchor with a href the page being mentioned
    • e-content - The content of the reply, likely a div (e.g. blog text)
    • u-url - The permalink of current page
    • p-author h-card - Containing element for details about the author of this page, likely a div
      • p-name u-url - On an anchor with href to a the author’s profile and text for authors name
      • u-photo - Image element of author’s avatar

You can hide any of these you like. If your html has the above set up right, you tell another page it has been mentioned and it will know how to find the details.

For a website to receive your webmention it needs a webmention endpoint. This is likely to be in the head as a link rel="webmention" but there are other possibilities. If this endpoint is known to you then it’s possible to send the webmention as simply as this:

curl -si https://site.com/webmentions/ -d source=https://yourpage.com/your-reply.html -d target=https://site.com/great-post.html

You likely wouldn’t do it like this, as you don’t want to hunt down the endpoint, but this is just to demonstrate how basic the webmention itself is. It’s just an HTTP POST to the webmention endpoint with 2 bits of data, the target of the mention, and the source.

If your source page has all the right html elements with the right class names then then they will receive a webmention with all the required info included. That’s it.

In real use, sending the mention you can use one of the available tools to do this, or write your own script. Webmention.app has an api or a commandline tool you can use. I am using Telegraph. These tools allow you (either manually or programmatically) to specify the source and target urls and they take care of some validation and webmention endpoint discovery.

Hugo Webmentions

In my case, I have hugo setup so that when I include certain properties in a markdown post, my templates render the relevant html for the webmention. For example:

---
title: "Street Party at a Distance"
date: 2020-05-08T22:08:37+01:00
draft: false
replyTo: https://trivial.observer/blog/avoiding-the-organised-fun/
replyToTitle: Avoiding the Organised Fun
---

Once a post with these properties has been published I go to Telegraph and provide the permalink of my new post and the link I’m mentioning. You could have this happen automatically at deploy time depending on your setup, but I like the manual approach.

The pain I had with this, was testing it, and I ended up using 2 online services. IndieWebify.me has some tests to validate your html including your h-entry, as does X-Ray. This didn’t cover everything, but I’ll get into that later.

Receiving Webmentions

You have a service that listens on an endpoint. When someone hits the endpoint with a source url and a target url, it will validate the urls and attempt to parse the source html. If it succeeds, you have a webmention.

There are various options for software you can use to receive webmentions for you. Such as stapibas, webmentiond, and the one I use, Webmention.io. Like the other 2 mentioned, you can host your own Webmention.io instance, but it is available as a free service.

If you sign up to webmention.io, and you include the following in your site’s <head> it will start parsing webmentions for you. As they come in, you’ll see them in your webmention.io dashboard.

<link rel="webmention" href="https://webmention.io/trivial.observer/webmention" />
<link rel="pingback" href="https://webmention.io/trivial.observer/xmlrpc" />

If you host your own webmention software elsewhere, then the links are different, but the method the same.

Displaying Webmentions

So this step is pretty generic in that it involves hitting an api and displaying the results on your website, and I’m not done with this step myself, but I do want to highlight some gotchas I had with my setup and webmention.io.

You can access the webmentions for your site from the api. For example:

curl https://webmention.io/api/mentions.jf2?target=https://trivial.observer/blog/avoiding-the-organised-fun/

This will give you a payload of all the webmentions you’ve been sent for that url.

{
    "type": "feed",
    "name": "Webmentions",
    "children": [
        {
            "type": "entry",
            "author": {
                "type": "card",
                "name": "Basil",
                "photo": "https://webmention.io/avatar/trivial.observer/ea976bb846a878732389d8a2db2727ea1032a02f556c2f8a262d57d1c45a8c0c.png",
                "url": "https://trivial.observer"
            },
            "url": "https://trivial.observer/blog/street-party-at-a-distance/",
            "published": "2020-05-08T22:08:37+01:00",
            "wm-received": "2020-05-09T13:45:20Z",
            "wm-id": 794609,
            "wm-source": "https://trivial.observer/blog/street-party-at-a-distance/",
            "wm-target": "https://trivial.observer/blog/avoiding-the-organised-fun/",
            "name": "Street Party at a Distance",
            "content": {
                "html": "<p>So today was the VE Day street party, the bunting was out, there were cakes and biscuits...trimmed for blog",
                "text": "So today was the VE Day street party, the bunting was out, there were cakes and biscuits as... trimmed for blog"
            },
            "in-reply-to": "https://trivial.observer/blog/avoiding-the-organised-fun/",
            "wm-property": "in-reply-to",
            "wm-private": false
        }
    ]
}

If you miss that trailing slash off the url you send to the api, you’ll get this response:

{"type":"feed","name":"Webmentions","children":[]}

And maybe you’ll piss away a good few hours trying to figure out why, maybe you wont.

So I hit this api, and I loop the response and write them out under my blog post:

$.getJSON("https://webmention.io/api/mentions.jf2?target=https://trivial.observer/blog/avoiding-the-organised-fun/", function(data) {
  data.children.forEach(mention => {
    //code here to write the various properties of mention to my page.
    //checking mention[wm-type]  "in-reply-to", "is-like-of", "is-bookmark-of", "is-repost-of". Displaying them differently.
  });
});

Getting Stuck

This is where I got stuck for over a day. All the online webmention markup testers said my website was marked up correctly, and I had managed to send a webmention to a page who’s webmention endpoint was using webmentiond. However, when I send a webmention from one of my pages to another one of my pages, webmention.io was not picking up the data from my page.

I ended up having to set up another website with a copy of my source page and repeatedly send a mention from it to my webmention.io endpoint, and every time it didn’t work, I’d strip something from the html that I thought might be causing the issue.

Since you can’t mention a page from the same source url twice, my test website ended up have 23 copies of a slightly different page. After about 4 hours, index23.html is the one that finally worked. The parent element of my h-entry element had a class called hfeed, this was breaking the webmention.io parser for some reason. Once removed sending a mention to to a webmention.io endpoint worked fine.

Getting Mastodon Interactions

In adition to all this, if you would like for your mastodon (or twitter, or whatever) interactions to be sent to your webmention.io dashboard, you can sign up your website to Bridgy. Once you sign up and link your Bridgy account to your mastodon account, when you toot a link to a blog post, any likes, boosts, or direct replies to that toot will be seen by Bridgy, translated into webmentions and sent to your way. They will appear in your webmention.io dashboard and will be returned by the api just like any normal mention.

Here’s an example of a like from mastodon in a webmention.io api response

{
    "type": "entry",
    "author": {
        "type": "card",
        "name": "Andy C",
        "photo": "https://webmention.io/avatar/cf.mastohost.com/5adb69fd0b47c24db0848648a9c6904201c25ffed3f6e45c484939f04fe785b3.jpg",
        "url": "https://mastodon.sdf.org/@andyc"
    },
    "url": "https://sarcasm.stream/@basil/104128600217477334#favorited-by-4",
    "published": null,
    "wm-received": "2020-05-08T16:10:52Z",
    "wm-id": 794329,
    "wm-source": "https://brid-gy.appspot.com/like/mastodon/@basil@sarcasm.stream/104128600217477334/4",
    "wm-target": "https://trivial.observer/blog/avoiding-the-organised-fun/",
    "like-of": "https://trivial.observer/blog/avoiding-the-organised-fun/",
    "wm-property": "like-of",
    "wm-private": false
}

That’s it

All of this is really just data plumbing. Set up your data right, ping an endpoint, endpoint collects it. Ask endpoint for data it’s collected, display on page. As ever, devil is in the details.

Most of my struggles were debugging webmention.io when it didn’t behave as expected. It ended up that the reports from submitting the test webmention via Telegraph proved to be the most useful in tracking down any issues.

Aside from the issue with the trailing slash on the api call and the parsing problem with my markup it was all quick and easy. Only other thing that took time (and is still going) is the logic of how I want to display the mentions on my page.