Speeding up the Azure DevOps Extension tasks

jessehouwing

Jesse Houwing

Posted on January 2, 2023

Speeding up the Azure DevOps Extension tasks

Speeding up the Azure DevOps Extension tasks

In particular the Publish Azure DevOps Extension task that spent quite a bit of time extracting the vsix prior to putting it back together.

A little background

Every Azure DevOps Extension is published using a vsix file. Basically a zip file with some additional manifest metadata. Before the Azure DevOps Extension Tasks were created I had created a very similar extension that had built-in support to update the version number of tasks prior to publishing. I later contributed that code to the Azure DevOps extension tasks.

tfx-cli, the utility used to publish extensions from the command-line has never gained said functionality, it can do all kinds of magic for the extension manifest, but not for the embedded task manifests. Plus, it can only perform its magic when publishing from source, not from vsix.

To support these features in the Azure DevOps Extension tasks, we basically extract the whole vsix, patch the manifest files and zip the vsix back up.

When building the Azure DevOps Task Zips scripts - in which I do a very similar thing to generate a copy of each built-in task that can be installed side-by-side the original version - I discovered a way to speed up the process. Instead of extracting the whole vsix file, I only extracted the files that need patching:

  • extension.vsixmanifest
  • extension.vsomanifest
  • **/task.json
  • **/task.loc.json

Then patch the manifest files and then update the original vsix file with the patched manifest files.

The result is that instead of extracting an archive of 31MB and 30k files, I now only have to extract 22 files. Instead of searching 30k files for manifest files, I only have to search those 22 files. And instead of compressing 81MB and 30k files again, I now only have to add 1 updated file.

Extracting archive: D:\a\1\vsix\vsts-developer-tools-build-tasks.vsix

Files: 21
Size: 81485
Compressed: 31328189

============================================================================

Scanning the drive:
29 folders, 21 files, 81287 bytes (80 KiB)

Updating archive: jessehouwing.vsts-developer-tools-build-tasks-dev-4.0.143.gen.vsix

Keep old data in archive: 4710 folders, 30155 files, 85539629 bytes (82 MiB)
Add new data to archive: 29 folders, 1 file, 2888 bytes (3 KiB)
Enter fullscreen mode Exit fullscreen mode

Log file showing the new process

The result that counts is that this process reduces the time to publish from a vsix file from 0:02:58 to 0:00:14. A 92% reduction in time!

Speeding up the Azure DevOps Extension tasks

When will you benefit from this time saving?

To gain these benefits you must:

  • Use the PublishAzureDevOpsExtension@4 task (only @4 will have this new feature).
  • Publish from vsix.
  • Use any of the override properties: extensionId, extensionName, extensionVersion, publisherId, extensionTag, updateTasksVersion, updateTasksId, extensionVisibility, extensionPricing.
  • (For now:) set the magic variable: PREVIEW_FAST_UPDATE = true.

Speeding up the Azure DevOps Extension tasks
Add a variable to your Azure DevOps Pipeline to enable this feature

This feature now works on all operating systems: Windows, Linux and MacOS.

Unfortunately, this isn't the only "slow" part of the extension publishing process, but it's a part I can speed up.

Feedback wanted

Please let me know whether this works for you! I'm interested in both success and failure reports. Please reply to the issue below:

And if you've been using the Azure DevOps Extension tasks, please consider:

💖 💪 🙅 🚩
jessehouwing
Jesse Houwing

Posted on January 2, 2023

Join Our Newsletter. No Spam, Only the good stuff.

Sign up to receive the latest update from our blog.

Related

Hey! You broke our pipeline!
azuredevopspipelines Hey! You broke our pipeline!

March 2, 2023

Speeding up the Azure DevOps Extension tasks
azuredevopspipelines Speeding up the Azure DevOps Extension tasks

January 2, 2023