AWS Graviton Weekly # 14: December 2nd, 2022 to December 9th, 2022

You're receiving this because you subscribed here OR here

Welcome to Issue # 14 of AWS Graviton Weekly, which will be focused on sharing everything that happened in the past week related to AWS Graviton: from December 2nd, 2022 to December 9th, 2022.

In the last issue, I shared some talks from AWS re:Invent 2022 related to Graviton and the AWS Silicon ecosystem, but many of the talks were not available yet in YouTube.

So, in this issue, I will provide an update to those, and I will let the summary of Peter DeSantis's keynote.

Still the same recommendation: Subscribe to the AWS Events YouTube channel and click in the bell to get notified every time they upload a new video.

Enjoy the content of this week. It's a lot, so take your notes.

News

NAB announces long-term cloud deal with AWS

NAB has also recently optimised costs for its cloud-based infrastructure. Using AWS services and products, including AWS Graviton, NAB implemented a program to save over $1 million per month in cloud costs, promoting a company culture focused on both efficiency and innovation for customers.

New Fortinet service offers next-gen firewall protection for AWS environments

Benefits include regionwide network protection at optimized costs, with FortiGate CNF designed to aggregate security across cloud networks, availability zones and virtual private clouds in a cloud region. FortiGate CNF natively supports AWS to optimize cloud security spending and uses AWS Graviton instances to deliver better price-performance than other offerings.

AWS launches Graviton3E, its new Arm-based chip for HPC workloads

At its traditional evening keynote at re:Invent, AWS tonight announced quite a bit of new hardware in its cloud, starting with a new version of its Nitro hypervisor, new instance types and a new version of its custom Arm-based Graviton chips, which was specifically designed for powering high-performance computing workloads. This new Graviton3E chip — a variant of the existing Graviton line — promises significant performance improvements, including 35% better performance for workloads that heavily depend on vector instructions.
These new chips will obviously power new AWS EC2 instance types, starting with the logically dubbed Hpc7G. This new instance type will come in a variety of sizes, with up to 64 vCPUs and 128 GiB of memory. It’ll take until early 2023 before these instances become available, though. For more network-intensive workloads, AWS is also launching a new Graviton3E instance type (c7gn).

AWS Makes Economic Case for Graviton Processors

Amazon Web Services (AWS) is making available in preview a C7gn instance on the Amazon Elastic Compute Cloud (Amazon EC2) based on AWS Graviton processors. The C7gn instances provide up to 200 Gbps network bandwidth and as much as 50% higher packet-processing performance compared to previous-generation C6gn instances.

Product update: Arm Development Studio 2022.2 now available

Arm has released the latest update to Arm Development Studio, with the 2022.2 Gold, Silver, and Bronze editions. The 2022.c Platinum Edition will follow soon after for users with access to that edition.

Articles and Tutorials

Let’s Architect! Optimizing the cost of your architecture, by Luca Mezzalira (Principal Serverless Specialist Solutions Architect at Amazon Web Services), Laura Hyatt (Solutions Architect II at Amazon Web Services (AWS)), Vittorio Denti (Machine Learning Engineer at Amazon), Zamira Jaupaj (Enterprise Solutions Architect at Amazon Web Services (AWS)), Ben Moses (AWS Senior Solutions Architect), and Michael Holtby (AWS Senior Manager Solutions Architecture)

Optimizing Trino using EC2 Spot instances with Zillow by Santhosh Venkatraman (Software Engineer at Zillow) and Rupesh Kumar Perugu (Software Engineer at Zillow)

In this installment of the Trino Summit 2022 sessions, we jump into an exciting topic by folks from Zillow about running Trino on spot instances. Spot instances are cheap and ephemeral nodes that lead to reduced overall compute costs. Spot instances are cheaper as they are not guaranteed to remain available. In this session, Zillow engineers talk about how they use Trino on spots to take advantage of the cost savings while handling the transitory nature of spots.

If you prefer the video, you can watch the talk here

How to Save 73% or More on Inference Costs, by Abhi Gupta (Senior Product Analyst at OctoML)

In machine learning, inferencing accounts. for 90% of AI/ML cloud spend. But there are ways to cut down on the cost of models running in production without impacting latency which degrades user experience. The OctoML team ran an analysis of three popular computer vision and NLP models, against two different CPU-based cloud hardware targets using the OctoML platform. OctoML achieved cost savings and speedups on all three models on both hardware types, but saw huge gains on AWS's Graviton3 powered instances.

VxWorks on Graviton2: Setting up a VxWorks instance in AWS

Optimizing for sustainability, by Liz Fong-Jones (Field CTO at honeycomb.io) in the keynote for DDD Brisbane 2022:

Slides from my keynote at DDD Brisbane on sustainability of IT and adoption of ARM64/Graviton! Your company can make a difference and reduce its environmental impact through a little bit of engineering effort!

Here are all the announcements from re:Invent 2022 related to AWS Silicon

Peter DeSantis's keynote
First, you need to watch the keynote from Peter DeSantis (Senior Vice President of AWS Utility Computing).

Why? Because he shared some incredible stuff there related to AWS Silicon:

First, he announced the new generation of AWS Nitro v5 (link to the exact moment in the YouTube video), which is a massive jump in performance from the previous generation: 2x transistors, 50% faster DRAM speed and a PCIe adapter that provides about twice the bandwith. What this means exactly? This new card is able to support 60% higher packet per second rate with a 30% reduction in packet latency and with 40% better performance per watt.

Then, he continued with the announcement of a new instance type called C7gn, a new network optimized EC2 instance, which is the first instance based in the Nitro v5 system and Graviton. The specs are simply insane: 200 Gbps per second of networking bandwitdth and up to 50% better packet processing performance (compared to C6gn instances). If you want to know more details, I encourage you to read this post from Jeff Barr here.

After that, he shared a very interesting number about Graviton3 based instances: they use up to 60% better performance per watt vs comparable x86 instances while providing 25% better performance compared to Graviton2 instances. Very cool, right?

And then, he shared the coolest stuff of the night: AWS built a new Graviton chip called Graviton3E, a variant of the Graviton3 processor but it's specifically optimized to do more floating points and vector math, which is perfect for HPC applications. Compared to Graviton3 based instances, it does 35% higher performance on HPL (a computational benchmark for lineal algebra), 12% on Gromacs and 30% better on common financial options models

To go even further, he announced another new instance called HPC7g, the lastest generation of EC2 instance for compute-intensive HPC workloads, powered by Graviton3E.

Then, he talked slightly about the AWS Trn1 instances, which is powered by up to 16 AWS Trainium accelerators purpose built to accelerate DL training, 512 GB of memory and 800 Gbps of networking bandwidth.

Another exciting stuff from his keynote was the announcement of AWS Lambda SnapStart, a new capability from Lambda to drastically reduce the cold start latency at no extra cost. To know more about it, you can watch this video about the topic

There are more cool and exciting stuff in the video. So, feel free to watch it entirely. You will love it.

Now, I will share all the sessions related to AWS Graviton, and below of every session, you will see a short description about it. Feel free to pick your favorite ones, grab a cup of coffee and enjoy.

AWS re:Invent 2022 Graviton sessions

ADM201: Ad technology innovation with NBCUniversal and FreeWheel, by Ryan McConville (EVP, Ad Platforms at NBCUniversal Media), Jeff Pinard (SVP & CIO, AdSales and Advanced Advertising, Product & Engineering at NBCUniversal Media), Xinyu Huang (Vice President of Data at FreeWheel) and Stephanie Layser(WW Head, Publisher Ad Tech Solutions @ AWS)

In this session, intended for ad technology, product, and engineering leaders, hear how one of the largest media companies in the world used AWS to reinvent their TV and video advertising workloads. Learn how NBCUniversal built a first-party data platform in the AWS Cloud to create unique audiences and expand addressability and developed privacy-enhanced solutions to collaborate with partners more effectively. Then see how FreeWheel, a Comcast company, improved performance for data and analytics workloads running at massive scale using AWS Graviton–based instances and other cloud capabilities. Learn about best practices, architectures, and lessons learned building scaled publisher-side ad technology workloads.

ADM302: Running high-throughput, real-time ad platforms in the cloud, by Victor Gershkovich (DevOps Team Lead at AppsFlyer), Gerry Louw (Head of WW Solutions Architecture - Advertising and Marketing at Amazon Web Services (AWS)) and Akhil Aendapally (WW Senior Solutions Architect - Advertising and Marketing at AWS)

In this session, designed for technology and engineering leaders, explore best practices and learnings from industry users for how AWS can reduce costs and improve performance for scaled real-time advertising workloads. Learn how AppsFlyer moved their real-time platform that handles more than 800 billion events per day into a scalable, cloud-native architecture using Amazon EKS and AWS Graviton-based instances to improve cost performance and reduce their carbon footprint. Then, hear AWS ad technology experts share customer stories and pitfalls to avoid when running ad platforms at scale.

CMP 201: Silicon innovation at AWS, by Ali Saidi (Sr. Principal Engineer at AWS)

Organizations are bringing diverse workloads onto AWS at a faster rate than ever before. To run diverse workloads with the performance and costs that users expect, AWS often innovates on their behalf and delivers breakthrough innovations even at the silicon level. AWS efforts in silicon design began with the AWS Nitro System but quickly extended to AWS Graviton processors and purpose-built inference chips with AWS Inferentia. In this session, explore the AWS journey into silicon innovation and learn about some of the thought processes, learnings, and results from the experience so far.

CMP 223-L: Compute innovation to enable any application in the cloud, by Dave Brown, (VP of EC2)

AWS offers the most comprehensive set of capabilities and continually innovates across infrastructure and services so you can build, run, and scale applications in the cloud, on premises, and at the edge. Join Dave Brown, VP of Amazon EC2, to hear about the innovations AWS is delivering for millions of organizations. Dave discusses how AWS has developed custom silicon optimized for the cloud and how you can take advantage of AWS compute innovations including processors, machine learning chips, and high-performance storage products.

CMP 301: Powering Amazon EC2: Deep dive on the AWS Nitro system, by Ravi Murty, Senior Principal Engineer, AWS

The AWS Nitro System is a rich collection of building block technologies—including AWS-built hardware offload and security components—that is powering the recent and future generations of Amazon EC2 instances with an ever-broadening selection of compute, storage, memory, and networking options. In this session, dive deep into the Nitro System, review its design and architecture, explore new innovations to the Nitro platform, and see how it has made the seemingly impossible possible.

CMP302: Confidential computing with AWS Compute by Arvind Raghu (Business Development & GTM Strategy, EC2 Core, Nitro Enclaves at AWS) and J.D. Bean (Principal Security Architect - Amazon EC2)

At AWS, confidential computing is defined as the use of specialized hardware and associated firmware to protect in-use customer code and data from unauthorized access. In this session, dive into the hardware- and software-based solutions that AWS delivers to provide a secure environment for customer organizations, with confidential compute capabilities such as the AWS Nitro System and AWS Nitro Enclaves. Additionally, learn about the building blocks for AWS Nitro Enclaves, including tooling, SDKs, attestation, and topics such as CI/CD.

[NOT AVAILABLE YET] CMP303: Get hands-on with AWS Graviton-based EC2 instances, by Arthur Petitpierre (EC2 & Graviton Principal Specialist SA, AWS) and Seth Fox ( Sr. Manager Solutions Architecture, AWS)

AWS Graviton processors are designed to deliver the best price performance for a wide variety of workloads in Amazon EC2. In this workshop, walk through moving a workload to AWS Graviton-based instances, including containerized applications. This workshop is great for developers or IT practitioners who are running Linux-based workloads in Amazon EC2 and are looking for better price performance. You must bring your laptop to participate.

[NOT AVAILABLE YET] CMP309: AWS Graviton adoption by Honeycomb.io using Amazon EC2 and AWS Lambda, by Sunil Ramachandra (Sr.Solutions Architect at AWS) and Pierre Tessier (Director of Solution Architects at Honeycomb.io)

CMP320: Accelerating semiconductor design, simulation, and verification, by David Pellerin (Head of Worldwide Business Development for Infotech/Semiconductor, Amazon Web Services), Mark Galbraith (VP Productivity Engineering at ARM) and William Cornwell (Vice President at Marvell Technology)

AWS offers many solutions to design, simulate, and verify the advanced semiconductor devices that are the foundation of modern technology. Electronic design automation (EDA) workloads are critical to the success of chip development. EDA requires computing for digital and analog simulations, logic synthesis, design rule checks, and physical verification. In this session, discover best practices for running HPC or EDA workloads on AWS using Amazon EC2. Hear from Arm about how they use AWS to accelerate EDA workloads in the cloud using Arm-based AWS Graviton instances. Also, Marvell shares how they’re using EDA in the cloud to scale their highly differentiated cloud-optimized silicon solutions for customers like AWS.

CMP325: Coding for multiple CPU architectures using lessons learned in HPC, by Arthur Petitpierre (EC2 & Graviton Principal Specialist SA, AWS) and Karthik Raman (HPC Application Engineer, Amazon)

This chalk talk details best practices and lessons learned about coding for multiple architectures (x86 and AWS Graviton) from the last ten years in the high performance computing (HPC) community. Amazon EC2 has seen a significant rise in the adoption of Arm-based AWS Graviton instances. When supporting multiple architectures, it’s important to ensure a consistent developer experience. When workloads run on multiple architectures, platform nuances and commonalities must be addressed to forge a transparent developer environment. Come learn how HPC development techniques can be applied to almost any compute workload and how to effectively code for multiple compute infrastructure architectures.

CMP 327: AWS Graviton deep dive: The best price performance for AWS workloads, by Sudhir Ramat (Senior Manager, EC2 Core Compute at AWS) and Oran Barak (Head Of Core Compute Engineering at Stripe):

From many major instance families in Amazon EC2 to managed services such as AWS Lambda, Amazon Aurora, and Amazon EKS, AWS Graviton-based instances are used by tens of thousands of customers to get significant price-performance benefits for a wide variety of workloads on AWS. AWS Graviton3 processors provide up to 25 percent better performance over AWS Graviton2 processors, which already provided significant price-performance benefits. This session dives deep into AWS Graviton3 processors, including suitable workloads and considerations for adoption, and it features an AWS customer speaking about their processor adoption experience.

The most interesting stats of this session? For Trino (SQL Engine), Stripe reduced their average latency by over 50% and reduced their overall error rate.

[NOT AVAILABLE YET] CMP 408: Running containerized workloads on AWS Graviton–based instances, by Seth Fox ( Sr. Manager Solutions Architecture, AWS) and Michael Fischer (Principal Specialist SA, AWS)

Containers are the easiest way to take advantage of AWS Graviton’s unparalleled price performance. In this chalk talk, learn how to build, test, and deploy containers for AWS Graviton using a multi-architecture approach that allows you the maximum flexibility to deploy anywhere on AWS with Amazon ECS, Amazon EKS, and AWS Fargate. The talk also provides techniques to help you migrate your existing containers to AWS Graviton.

[NOT AVAILABLE YET] CMP409: Enhancing OS and application security on AWS Graviton–based instances, by William Koplitz (Senior Specialist Solutions Architect - Graviton, AWS) and Arthur Petitpierre (EC2 & Graviton Principal Specialist SA, AWS)

AWS Graviton processors feature key capabilities that enable you to run cloud-native applications securely and at scale. AWS Graviton3 processors feature always-on memory encryption, dedicated caches for every vCPU, and support for pointer authentication. In this chalk talk, dive deep into how developers can build secure applications using pointer authentication.

PRT220: How to maximize HPC productivity, performance, and portability , by John Linford (Technical Product Manager CPU Software, NVIDIA)

In this session, learn how NVIDIA enables high-performance computing (HPC) on over 200 Amazon EC2 instance types. NVIDIA provides a multi-platform, standards-compliant, vendor-supported solution for HPC application development that supports all major programming models. Its proven compilers, libraries, and software tools support AWS Graviton to maximize developer productivity, enable hardware platform choice, and facilitate an optimal price-performance solution for HPC applications in the cloud.

PRT246: Building real-time applications to utilize AWS Graviton by Tal Gabay ( Engineering Manager, Wix.com), Raj Pai (Vice President, AWS), Subbu Iyer (President and CEO, Aerospike), and Kyle Bush( VP, Data Analytics Architecture, Fidelity Investments)

With the growth of real-time data surpassing every other category, application architects are looking into more efficient infrastructure and real-time data platforms on which to build new real-time applications. AWS Graviton with Aerospike Database provides an ideal starting point to build these next-generation services. In this session, Aerospike CEO Subbu Iyer discusses how to design real-time applications using the smallest cluster footprint and native support for Graviton. Subbu also reviews case studies from Fidelity Investments and Wix.com to illustrate real-time application design on AWS.

DOP315: Sustainability in the cloud with Rust and AWS Graviton with Emil Lerch (Principal DevOps Specialist at AWS) and Esteban Kuber (Principal Software Engineer at AWS)

AWS is focused on efficiency and innovation across its global infrastructure, and that focus highlights how the decisions developers make can have an impact on energy consumption and green initiatives. In this session, learn about the benefits of Rust and AWS Graviton that can reduce energy consumption and increase productivity.

XNT401: Boosting .NET application performance with Arm64 and AWS Graviton3, by Sreelaxmi Pai (Sr. Modernization Architect, AWS) and Kirk Davis (Principal Solutions Architect, AWS)

Did you know you can achieve 40 percent or higher price performance for your .NET 6+ applications using Arm64 and AWS Graviton3 processors? This session dives into the details on how to compile ASP.NET Core for Arm64 and deploy to Graviton3. Learn how to automate the deployment process with an Arm64-based CI/CD pipeline and benchmark a sample application on both Graviton3 and x86 instances to illustrate the benefits. Also, explore strategies such as threading approaches versus those used in x86-based instances and other optimization techniques that you can use to optimize your .NET 6+ applications on Graviton3.

PRT224: Overcoming migration hurdles with CircleCI and AWS Graviton, by Charity Majors (CTO, Honeycomb.io), Tom Trahan ( VP Business Development, CircleCI) and William Koplitz (Senior Specialist Solutions Architect - Graviton, AWS)

Employing intelligent automation throughout the software development workflow is essential to avoiding continually escalating effort and costs as you grow, but not knowing how to do so may be holding you back.
This problem can be daunting, but luckily, with the right solutions that doesn’t have to be the case. Hear from Honeycomb, CircleCI, and AWS Graviton on the biggest obstacles teams face when scaling in the cloud and the DevOps and CI/CD practices companies like Honeycomb and Sweetgreen implemented with CircleCI and AWS Graviton, which led to 50 percent increases in speed and 50 percent decreases in costs.

PRT265: What to know before adopting Arm: Lessons learned at Datadog , by Jason Yee ( Staff Technical Evangelist, Datadog)

Moving from x86-based Amazon EC2 instances to AWS Graviton Arm-based processors can save you a lot of money—up to 40 percent better price performance. But can you just update your AWS CloudFormation templates from T2 to T4g and reap the savings? Datadog runs tens of thousands of nodes and has migrated a significant portion of their workloads to AWS Graviton. In this session, hear the top lessons Datadog has learned to help you prepare for a migration. Find out what changes you may need to make to your applications and how to operate and observe your services to get the most performance from the AWS Graviton architecture.

CON308: Reduce your operational and infrastructure costs with Amazon ECS, by Vibhav Agarwal (Sr PM, AWS Containers) and Francis Pereira (VP, Infrastructure Engineering at CleverTap)

In this session, walk through how to reduce operational overhead from the control plane with Amazon ECS. Learn about how to use containers for bin-packing workloads, efficient scaling techniques, cost savings plans, AWS Copilot, blueprints, and AWS Graviton.

Again: there are a lot of videos that have not being uploaded yet to the channel. So, subscribe here.