Easily use Firehose API on Bluesky Social with Dart and Flutter

As already noted by many media articles, Bluesky Social is a decentralized social networking service with a Twitter-like UI. Bluesky Social is built on a next-generation common infrastructure called the AT Protocol, and the road ahead is very ambitious and full of unknown possibilities.

Now, I will introduce the Firehose API, one of the APIs related to Bluesky Social, and how to use it easily with the Dart language. Bluesky Social's Firehose API is a very powerful Stream API that allows you to retrieve almost any event occurring on a particular server in real time.

What Is Dart?

For those who are not familiar with the Dart language, Dart is a relatively new programming language developed by Google. Dart has a syntax similar to Java and C, but with modern language specifications such as null safety. Above all, Flutter is probably the reason Dart is getting the most attention.

Dart programming language | Dart

Dart is an approachable, portable, and productive language for high-quality apps on any platform.

dart.dev

Flutter - Build apps for any screen

Flutter transforms the entire app development process. Build, test, and deploy beautiful mobile, web, desktop, and embedded apps from a single codebase.

flutter.dev

Well, there is already a powerful package that makes it easy to use Bluesky Social's API with Dart and Flutter, and this article will present sample code using that package!

Install Package

To easily use Bluesky Social's API in Dart or Flutter, use the package bluesky. I develop and maintain this package and it already supports almost all endpoints, is well tested and very stable.

It is already being used in several third-party apps, and I highly recommend this package for your Bluesky Social-related apps in Dart or Flutter.

bluesky | Dart package

The most famous and powerful Dart/Flutter library for Bluesky Social.

pub.dev

Or you can see more details in in following official page.

AT Protocol and Bluesky Social Things for Dart and Flutter | atproto.dart

Powerful suite of AT Protocol and Bluesky-related packages for Dart/Flutter

atprotodart.com

Well, let's install this package with following commands.

With Dart:

dart pub add bluesky

dart pub get

With Flutter:

flutter pub add bluesky

flutter pub get

And let's check your pubspec.yaml in your Dart or Flutter app. It is successful if it looks like this and this article uses bluesky v0.5.7.

name: bluesky_firehose
description: A sample command-line application.
version: 1.0.0
environment:
  sdk: ^3.0.0

dependencies:
  bluesky: ^0.5.7

dev_dependencies:
  lints: ^2.0.0
  test: ^1.21.0

Let's Use Firehose API

Now all that is left is to actually use the Firehose API with the bluesky package! The bluesky package makes it easy to connect to Bluesky Social's Firehose API as follows.

import 'package:bluesky/bluesky.dart';

Future<void> main() async {
  final bluesky = Bluesky.anonymous();

  final subscription = await bluesky.sync.subscribeRepoUpdates();

  await for (final event in subscription.data.stream) {
    event.when(
      commit: (data) {
        for (final op in data.ops) {
          print(op.uri);
          print(op.record);
        }
      },
      handle: print,
      migrate: print,
      tombstone: print,
      info: print,
      unknown: print,
    );
  }
}

Very simple, right?

If you dart run the above Dart program, you will see the events occurring in bsky.social streaming in real time like:

at://did:plc:cojuqgrp6syogwzytq3p5zdq/app.bsky.graph.follow/3jxsejsfjas2n
{$type: app.bsky.graph.follow, subject: did:plc:pwm3nal7z4sht4id6wniiahv, createdAt: 2023-06-10T08:53:45.050Z}
at://did:plc:vopt5zssk6uotcsgnimyuiy3/app.bsky.feed.like/3jxsejsgnev2t
{$type: app.bsky.feed.like, subject: {cid: bafyreihoge573pwtux4dgedovmtitnwdnibvekrr7nvqs3qjbnrplnsvxq, uri: at://did:plc:awnhnq23opcufc2rnqbggjnp/app.bsky.feed.post/3jxsdpbessq2s}, createdAt: 2023-06-10T08:53:44.841Z}
at://did:plc:njyxpfp2rzb7addisbq74epa/app.bsky.graph.follow/3jxsejskzz72r
{$type: app.bsky.graph.follow, subject: did:plc:uo2g6reoqnrudyoeotwveuc2, createdAt: 2023-06-10T08:53:45.453Z}
at://did:plc:b4td4mpmhj76dwdqrds72p4p/app.bsky.feed.like/3jxsejsnh5q2h
{$type: app.bsky.feed.like, subject: {cid: bafyreienlb76pav7d4gvdpy5opmllldhjongwnmmkkrjof5cg4olawe4he, uri: at://did:plc:sho65umi6t4ohqyaijutbdcr/app.bsky.feed.post/3jxs56p2ost2l}, createdAt: 2023-06-10T08:53:45.491Z}
at://did:plc:64kdoplv62s6veonqincpgod/app.bsky.feed.post/3jxsejsib5h2r
{text: How long was this in here for that this happened?, $type: app.bsky.feed.post, embed: {$type: app.bsky.embed.recordWithMedia, media: {$type: app.bsky.embed.images, images: [{alt: , image: {ref: {$link: bafkreiddranx4ph7m53cdpiwme5g4zjuo4djaapdychlc6nm46ogyod2zi}, size: 876186, $type: blob, mimeType: image/jpeg}}]}, record: {$type: app.bsky.embed.record, record: {cid: bafyreifbe7p5fozzx4ualnrspqqfsfs2xoas4r2ambxjstcdkji3i7tq4e, uri: at://did:plc:64kdoplv62s6veonqincpgod/app.bsky.feed.post/3jxsdq3tgma2h}}}, createdAt: 2023-06-10T08:53:45.372Z}
at://did:plc:xopjsprtk24yvyfyfzfifxki/app.bsky.graph.follow/3jxsejt2nz72r
{$type: app.bsky.graph.follow, subject: did:plc:cpogq3zcrybwsibq4hzjnbxa, createdAt: 2023-06-10T08:53:45.495Z}
at://did:plc:hgb7kmelvhfdhlc777diyokk/app.bsky.feed.like/3jxsejunxmh2r
{$type: app.bsky.feed.like, subject: {cid: bafyreiayanec5xmjfs3a2ey5bu3onrwbuz53i6w5sd3k55bmnh4444iuiq, uri: at://did:plc:5fwxaoha4lvtoseohazxr5uh/app.bsky.feed.post/3jxs7uqur2n2u}, createdAt: 2023-06-10T08:53:47.609Z}
at://did:plc:djarlkihrgk7uvwcf4umikno/app.bsky.graph.follow/3jxsejuudpi2h
{$type: app.bsky.graph.follow, subject: did:plc:dqvv2jev7pm4a66iozhzqehq, createdAt: 2023-06-10T08:53:48.522Z}
at://did:plc:oqbijttmayqui2xv3e2xwcdz/app.bsky.feed.post/3jxsejvcwj32l
{text: I’m just warming them up for you fam, $type: app.bsky.feed.post, reply: {root: {cid: bafyreibhc74rccmajeddna5xkps4iqwamwngf5ihlzeimaxbw7mzoj45ya, uri: at://did:plc:rlfsrnc4kvcgatjaykqvxfql/app.bsky.feed.post/3jxs3ehmevb2n}, parent: {cid: bafyreibhc74rccmajeddna5xkps4iqwamwngf5ihlzeimaxbw7mzoj45ya, uri: at://did:plc:rlfsrnc4kvcgatjaykqvxfql/app.bsky.feed.post/3jxs3ehmevb2n}}, createdAt: 2023-06-10T08:53:48.176Z}

However, for those who are not familiar with Bluesky's API or Dart, this code may seem complicated, so I will explain the main points.

First, to access Bluesky Social's API using the bluesky package, import package:bluesky/bluesky.dart and instantiate the Bluesky object as follows.

import 'package:bluesky/bluesky.dart';

Future<void> main() async {
  final bluesky = Bluesky.anonymous();
}

Since the Firehose API used this time does not require user authentication to access it, now we created an instance of the Bluesky object using the .anonymous() constructor, which does not require authentication. The .fromSession() constructor must always be used when using endpoints that require user authentication, but is not used in this article.

The next thing we have to do is to get the Firehose API Stream from an instance of the Bluesky object as follows.

final subscription = await bluesky.sync.subscribeRepoUpdates();

With just this one simple line you can get a Stream of Bluesky Social's Firehose API. This Stream is a long-lived connection and is communicated via WebSocket. But you need not be aware of these difficult communications at all, just write one line above and you will be fine.

Returned from subscribeRepoUpdates() is a subscription data representing the connection to the Firehose API. You can retrieve the data flowing from stream in the following way.

final subscription = await bluesky.sync.subscribeRepoUpdates();

await for (final event in subscription.data.stream) {
  print(event);
}

Like above, you can get events that occur in bsky.social from stream. But, the structure of event that can be obtained from this stream is not simple. This event object is of type Union, which simply means that there are multiple structures of objects flowing from the stream.

Do you have to handle those complex structures yourself? No, that is not necessary. The bluesky package provides easy handling of objects of type Union.

Not only in this example, but any object of type Union in the bluesky package can easily handle any structure that may flow as a Union using .when().

final subscription = await bluesky.sync.subscribeRepoUpdates();

await for (final event in subscription.data.stream) {
  event.when(
    commit: print,
    handle: print,
    migrate: print,
    tombstone: print,
    info: print,
    unknown: print,
  );
}

For example, using event.when() in the above example, you can safely handle objects with the different structures commit, handle, migrate, tombstone, info and unknown. The unknown is called when there is a Union structure not supported by the bluesky package, and is passed JSON (Map) that is not parsed to a specific object.

But, only the commit and handle patterns are mainly used at this time, and the other structures are not implemented or used in the official Bluesky API. So, let's focus on commit and handle now.

The commit event is fired when a post is posted on Bluesky, or when a specific post is liked and etc. A commit event consists of one or more ops, each op having an action that triggered the event to occur as follows.

create: When a specific record is created.
update: When a specific record is updated.
delete: When a specific record is deleted.

The Dart program allows you to do the following.

final subscription = await bluesky.sync.subscribeRepoUpdates();

await for (final event in subscription.data.stream) {
  event.when(
    commit: (data) {
      for (final op in data.ops) {
        switch (op.action) {
          case RepoAction.create:
            print(op.uri);
            print(op.record);
          case RepoAction.update:
          case RepoAction.delete:
        }
      }
    },
    handle: print,
    migrate: print,
    tombstone: print,
    info: print,
    unknown: print,
  );
}

For example, in the above case, from op.uri you can get the AT URI of a specific record, and from op.record you can get a specific record according to the AT URI and action.

Next, the handle event is fired when a particular user changes the handle on an account. The handle event is not as difficult as the commit event introduced earlier, you just can get the modified contents of the handle as follows.

final subscription = await bluesky.sync.subscribeRepoUpdates();

await for (final event in subscription.data.stream) {
  event.when(
    commit: print,
    handle: (data) {
      print(data.did);
      print(data.handle);
    },
    migrate: print,
    tombstone: print,
    info: print,
    unknown: print,
  );
}

Let's Handle Commit More Easily

I have shown you how to use Bluesky Social's Firehose API using the bluesky package, but there is actually an easier way to handle the commit event. It is to use RepoCommitAdaptor for the commit event as follows.

final subscription = await bluesky.sync.subscribeRepoUpdates();

// Add this
final repoCommit = RepoCommitAdaptor(
  onCreatePost: (data) {
    print(data.author);
    print(data.uri);
    print(data.record);
  },
  onUpdateProfile: (data) {
    print(data.record);
  },
  onDeleteFollow: (data) {
    print(data.uri);
  },
);

await for (final event in subscription.data.stream) {
  event.when(
    commit: repoCommit.execute, // And execute like this
    handle: print,
    migrate: print,
    tombstone: print,
    info: print,
    unknown: print,
  );
}

You can see at a glance that the RepoCommitAdaptor can be used to process each event in a very concise manner compared to handling each event individually using ops to action, etc., as in the previous example.

The above example specifies onCreatePost, onUpdateProfile, and onDeleteFollow when creating an instance of RepoCommitAdaptor, which means that only the following events can be filtered and retrieved.

onCreatePost: When the Post is created.
onUpdateProfile: When the Profile is updated.
onDeleteFollow: When the Follow is deleted.

Of course, the above examples are only some of the events supported by RepoCommitAdaptor.

The RepoCommitAdaptor provides very powerful and secure handling when using Bluesky Social's Firehose API with the bluesky package. When using subscribeRepoUpdates() to handle commit events, it would be best to use the RepoCommitAdaptor.

Conclusion

This is how to easily use Bluesky Social's Firehose API with Dart and Flutter using the bluesky package.

This article is basically an example with a Dart program, but the bluesky package also works with Flutter, so similar code can be used to make the Firehose API work in a Flutter app. You can use the StreamBuilder widget when working with Firehose API Streams in Flutter.

As mentioned earlier, the bluesky package also supports many endpoints other than the Firehose API presented here. This is a very powerful package that will surely be useful in the Dart/Flutter apps you develop!

In addition to the bluesky package I have presented here, I have also developed many AT Protocol related packages for Dart/Flutter in the following monorepo. If you are interested in AT Protocol related packages for Dart/Flutter, please check it out!

myConsciousness / atproto.dart

🦋 AT Protocol and Bluesky things for Dart and Flutter.

AT Protocol and Bluesky Social Things for Dart/Flutter 🦋

Welcome to atproto.dart 🦋

This project will maximize your development productivity about AT Protocol and Bluesky things.

Give a ⭐ on GitHub repository and follow shinyakato.dev on Bluesky!

1. Motivation 💪

AT Protocol and Bluesky are awesome.

This wonderful platform needs a standard and highly integrated SDK atproto.dart provides the best development experience in such matters for Dart/Flutter devs.

2. Packages & Tools ⚒️

2.1. Dart Packages

Package	pub.dev	Docs
at_identifier: core library for the syntax in the AT Protocol standard		README
nsid:

…

View on GitHub

Suggestions for improvements and pull requests are also very welcome. Or you can contact me on Bluesky :)

https://bsky.app/profile/shinyakato.dev

Thank you.

Blog