I think my first objective is going to be extraction of posts so that they can be added to a repository. To do this I need to understand the data I am getting, but also keeping in mind the main goal of publishing and updating articles.

Pull Missing Data

When grabbing articles it looks like the series assigned is not provided. This isn't a problem because updating does not need to specify all data.

You receive the organization, but putting an article needs tho organization ID.

Meta Duplication

Dev original placed meta data within the content itself, front matter, this is still supported, and takes precedence.

This isn't a problem, either the user manages the content or this library could allow normalization. I wonder if dev.to would have issue with an API update only doing partial normalization (recall my profession is QA).

Update the Changed

Publishing to an external system is going to be a challenge. See the usual CI/CD is to create a package and push out the whole thing.

In the case of dev.to this would mean updating every article every commit (or arbitrary rule). This isn't exactly fair or the expected use of the api.

How did the implementation I originally mentioned handle this? It provided a separate file to specify what the CI should publish. I expect items modified would enter this list and then leave.

What is a better way? Well git will store file modification timestamps, this could then be queried from the file system. However this needs to be compared with a last published date.

Dev.to will provide to modification time, however this being REST and not graphQL this means pulling all data, almost as bad as publishing all data.

Pipelines are not designed to modify the repository, for good reason. Thus having the publish step archive its work is not really in the cards.

So what is the answer? We need to establish a point of publication, tags are a good way to do this. If we publish on a tag the git can be asked for the files changed between it and the previous tag.

But this has a problem. What if publication fails? The next tag expects the previous tag to have published. At this point my thoughts are to request the failed tag be removed from the server. Which has its own problems. Git won't delete tags from a repository like it does with branches and --prune. We also could create several local branches and push them up and only the last change set would publish.

Promotion

Concider following me if you liked the article. Check out my profile to see if you enjoy my other writings. I cover git, testing, D, and more.