I want to learn more about how to work with "fediverse" tech — Mastodon, ActivityPub, WebFinger, etc.
$ mastopost --text "Hi. This is some stupid shit I wanna say."
One thing I soon hope to do is set up my own microblog that mirrors its posts to Mastodon, rather than posting directly to Mastodon, reflective of a kind of local-first, steward-your-own-stuff ethos.
An instant-gratification way to learn to do something is to write a useful script that does it. So, yesterday I wrote my first version of a tool called mastopost. This version successfully posted text, but did not implement support for media attachments.
Basic status posting
With the help of a post by Roy Tang, using Li Haoyi's wonderful libraries requests-scala and pprint, this was super simple. The relevant Mastodon API docs are here.
The heart of the script was initially just:
import java.net.URLEncoder
// images not yet supported
case class Config( text : String, url : String, accessToken : String, images : List[String], verbose : Boolean)
def pathJoin( a : String, b : String ) : String =
val normA = if a.last == '/' then a.init else a
val normB = if b.head == '/' then b.tail else b
s"${normA}/${normB}"
// the return value will become the exit code of our script
def post( config : Config ) : Int =
val statusEndpoint = pathJoin( config.url, "api/v1/statuses/" )
val headers = Map (
"Authorization" -> s"Bearer ${config.accessToken}",
"Content-Type" -> "application/x-www-form-urlencoded",
)
val formData = s"""status=${URLEncoder.encode(config.text, "UTF-8")}"""
requests.post( statusEndpoint, data=formData, headers=headers )
0 // if it didn't fail with Exception, send a good exit code!
This works fine!
You do have to create an access token. Go to the "Preferences" page of your logged-in Mastodon instance, then select "Development", then press the "New Application" button and fill out the form. When you are done, you will have an access code that will work!
Eventually my post(...) method grew, only because I wanted to be able to verbosely inspect responses, including bad responses.
// the return value will become the exit code of our script
def post( config : Config ) : Int =
val statusEndpoint = pathJoin( config.url, "api/v1/statuses/" )
val headers = Map (
"Authorization" -> s"Bearer ${config.accessToken}",
"Content-Type" -> "application/x-www-form-urlencoded",
)
val formData = s"""status=${URLEncoder.encode(config.text, "UTF-8")}"""
// we catch failures resulting from bad status codes, rather than just
// fail with exception, so we can print more informative errors
val response =
try requests.post( statusEndpoint, data=formData, headers=headers )
catch
case rfe : requests.RequestFailedException => rfe.response
if config.verbose then System.err.println( pprint( response ) )
if response.statusCode == 200 then
if config.verbose then System.err.println("Post succeeded.")
0 // good exit code
else
val rt = response.text()
val rtPart = if rt.isEmpty then "no response text." else s"response text: ${response.text()}"
System.err.println(s"Attempt to post failed! Status code ${response.statusCode}, ${rtPart}")
9 // arbitrary bad exit code
A beautiful command line
The rest of this first version of the script is just command-line parsing.
I really enjoy Ben Kirwin's library decline. It lets me write extremely tight code to validate and parse the command line, and gives me beautiful usage messages like this:
Usage: mastopost --text <string> [--url <string>] [--access-token <string>] [--image <string>]... [--verbose]
Posts toots to Mastodon.
Options and flags:
--help
Display this help text.
--text <string>, -t <string>
The text of your toot.
--url <string>, -u <string>
The URL of the instance to which you wish to post.
--access-token <string>, -a <string>
The access token to authenticate under.
--image <string>
The file or URL of images to attach.
--verbose
Print more information to the console.
Environment Variables:
MASTO_INSTANCE_URL=<string>
The URL of the instance to which you wish to post.
MASTO_ACCESS_TOKEN=<string>
The access token to authenticate under.
The command-line parsing code is reproduced below.
import com.monovore.decline.*
import cats.implicits.* // for mapN
case class Config( text : String, url : String, accessToken : String, images : List[String], verbose : Boolean)
val urlHelp = "The URL of the instance to which you wish to post."
val accessTokenHelp = "The access token to authenticate under."
val text = Opts.option[String] ("text", short="t", help="The text of your toot.")
val url = Opts.option[String] ("url", short="u", help=urlHelp) orElse Opts.env[String]("MASTO_INSTANCE_URL", help=urlHelp)
val token = Opts.option[String] ("access-token", short="a", help=accessTokenHelp) orElse Opts.env[String]("MASTO_ACCESS_TOKEN", help=accessTokenHelp)
val images = Opts.options[String]("image", help = "The file or URL of images to attach.").orEmpty
val verbose = Opts.flag("verbose", help="Print more information to the console.").orFalse
val allOpts = (text, url, token, images, verbose).mapN( (t, u, at, i, v) => Config(t,u,at,i,v) ) // a bit annoying
// Insert the mastodon post code from above here...
val command = Command(name="mastopost", header="Posts toots to Mastodon.")( allOpts )
command.parse(args.toIndexedSeq, sys.env) match
case Left(help) =>
println(help)
System.exit(1)
case Right( config ) =>
val exitCode = post( config )
System.exit(exitCode)
The command.parse(...) line yields an Either, which gives me my Config object if everything's cool, or the usage text to print if it is not.
Note how nicely decline supports values that can be supplied either as command-line options or as enviroment variables. It enforces that instance URL and access token must be supplied, but accepts them from either source.
Be careful though! decline won't find environment variables unless you explicitly supply the environment you want (as a Map[String,String]) in the call to command.parse(...). Usually you will just supply sys.env. This tripped me up for a few minutes.
Supporting media attachments
To support media attachments, you
- Hit a separate API endpoint to upload your media (up to four)
- Parse out the
idfrom the JSON response(s) of the media upload(s) - Include a
media_idsfield in your status post, containing an array of theid, treated as strings, not numbers
As I write, this is the current version of my script.
Uploading media turned out to be very easy. Uploads are supposed to be in multipart/form-data format, which requests-scala supports painlessly. The image is uploaded as of the fourth statement below. Again, the method is long only because I wanted to support detailed tracing of bad outcomes when the --verbose flag is set.
We have to decode a JSON response, which we do with Li Haoyi's ujson library. (See upickle for docs.)
Note that a success in Mastodon's API for uploading media can be the usual good HTTP response code 200, or also 202 for media that may take longer to process. That's why our test for success is
response.statusCode / 100 == 2
rather than
response.statusCode == 200.
def postMedia( config : Config, mediaSource : InputStream, filename : String ) : String = // returns the ID of the new media object
val mediaEndpoint = pathJoin( config.url, "api/v2/media" )
val multipart = requests.MultiPart( requests.MultiItem("file", data=mediaSource, filename=filename) )
val headers = Map (
"Authorization" -> s"Bearer ${config.accessToken}",
)
val response =
try requests.post( mediaEndpoint, data=multipart, headers=headers )
catch
case rfe : requests.RequestFailedException => rfe.response
if response.statusCode / 100 == 2 then
if config.verbose then
System.err.println(s"Media upload succeeded:")
System.err.println(pprint(response))
System.err.println()
var jsonOut = ujson.read(response.text())
jsonOut.obj("id").str // the id comes back as a JSON *String*, not a number
else
val rt = response.text()
val rtPart = if rt.isEmpty then "no response text." else s"response text: ${response.text()}"
val msg = s"Upload of media '${filename}' failed! Status code ${response.statusCode}, ${rtPart}"
System.err.println(msg)
if config.verbose then System.err.println(pprint(response))
throw new Exception(msg)
I wrote this in terms of InputStream rather than files, because I wanted to support URLs as media sources. It annoys me to have to manually download images, then reupload, when I want to attach an image from the internet as a status.
Once we have this method, we just have to turn our media filenames and/or URLs into InputStreams with some filename, hit the method, then post our status as we did before, except with a new array field called media_ids.
I had a hard time making this work, though, hitting the status with a form-data-encoded request as we did before. However I tried to encode media_ids, I couldn't get media to attach, even after reviewing Mastodon's documentation of its conventions for form data.
Thanks to Chris Jones, who also had a hard time getting form-data-encoded requests to attach media, I learned that you can hit these endpoints with JSON rather than form data. media_ids worked just fine with a JSON-ifed version of post(...):
// the return value will become the exit code of our script
def post( config : Config ) : Int =
val mediaIds = prepareMedia( config )
val statusEndpoint = pathJoin( config.url, "api/v1/statuses/" )
val headers = Map (
"Authorization" -> s"Bearer ${config.accessToken}",
"Content-Type" -> "application/json",
)
val jsonData =
val obj = ujson.Obj(
"status" -> ujson.Str(config.text),
"media_ids" -> ujson.Arr( mediaIds.map( ujson.Str.apply )* ),
)
ujson.write(obj)
// we catch failures resulting from bad status codes, rather than just
// fail with exception, so we can print more informative errors
val response =
try requests.post( statusEndpoint, data=jsonData, headers=headers )
catch
case rfe : requests.RequestFailedException => rfe.response
if config.verbose then System.err.println( pprint( response ) )
if response.statusCode == 200 then
if config.verbose then System.err.println("Post succeeded.")
0 // good exit code
else
val rt = response.text()
val rtPart = if rt.isEmpty then "no response text." else s"response text: ${response.text()}"
System.err.println(s"Attempt to post failed! Status code ${response.statusCode}, ${rtPart}")
9 // arbitrary bad exit code
There's a bit of busy-work getting from arguments to InputStream plus filename (which I bundle together as MediaStream in the script), but it's straightforward if slightly more verbose than I had hoped.
Anyway, please check out the latest evolving version of the script for more details.
Miscellaneous
Since the initial version, I've modified the image command-line argument to be media instead, since in theory you might want to upload videos and stuff.
Right now --text is a required argument to the script, but it's legal to have media-only posts. Sometime soon, I'll try to modify the script to support that.
For now, I'm going to keep MASTO_INSTANCE_URL and MASTO_ACCESS_TOKEN set in my environment, and posting will just be
$ mastopost --text "Hi. This is some stupid shit I wanna say."
along with some optional stupid --media.