Mastering The Heap: How to Capture and Store Images from Fetch Responses

wearypossum4770

Stephen Smith

Posted on May 2, 2024

Mastering The Heap: How to Capture and Store Images from Fetch Responses

As software developers we at times do not invest the time to understand how things work.

We always hear the adage "Don't reinvent the wheel."

This mindset can stifle creativity and a failure in advancement of our technical knowledge. This will result in less rewarding and fulfilling personal and professional lives.

To build up my previous post where our API presented an image to the browser, we will attempt to look into fetching and saving an image from an API. Why do we need this? We don't, but why not 🤷🏿.

There are many posts related to this topic, and not many approach the topic in a runtime agnostic fashion. Many examples use the DOM (Document Object Model) to accomplish this task. Today we will use a set of JavaScript runtime agnostic utilities; let's get started.

When in doubt READTHEDOCS

TLDR;

Security Considerations:

While we will attempt to use best practices, any code should be scrutinized. Do not use this in production without a approval from your organization, pentesting (penetration testing) to verify it follows your organizations safety guidelines.

Requisite Knowledge:

  1. Data encoding (Hexadecimal, base64, ascii, utf-8, utf-16).
  2. Fetch API (outside libraries will require additional steps).
  3. System I/O (input/output) calls.
  4. Asynchronous vs Synchronous programming.
  5. Types of media (image, audio, and video)

The Setup

  • Install depencies.
    1. curl -fsSL https://bun.sh/install | bash
  • Fetch/AJAX.
    1. Native Fetch
    2. node-fetch.
    3. axios
  • Setup a bun-flavored project.

    mkdir sample
    cd sample
    bun init
    
  • Setup Testing

    • create a feature folder to house our testing. This should help prevent your shell's autocomplete suggestions when running the "test" command.
    mkdir features
    
    • sample test module. Example Test File

Technology

The Code

Helper Function

  1. Change from header keys from kebab-case to camelCase.

    • Package Dependents Change Case
    • DIY
      • Create a replacer function that will handle our results from the String.replace function. Change Case
      • create the case change function Header to camelCase
      • create a function to convert case for the header's keys. Process Headers
  2. Create a jpeg image processing function. The call to the API will return a JPEG/JPG image, so we will not worry with other images.
    Process Image

  3. Create a helper function to extract the content disposition header.
    Get Filename

Fetching The Data

  • Prepare the object to request a new random image. Fetch Image
  • Make network request. We will be reading the raw data from the response. I do not recommend using any other library as you will have to contend with encoding as well as creating a Buffer, Blob, or TypedArray. So lets just cut the extra steps out. Fetch Data

Save The Data

  • the penultimate step is to save the data. You will have to figure this part out on your own (It's just import statements, and calling the function). I am not gonna give you everything silly. Save Image
  • The last step is to open the image and see what you got.

Testing

Maybe Later

Example Code

import { createWriteStream } from 'fs'
class FileNameError extends Error {
    constructor(filename:unknown) {
      super(
        `Incorrect data give for file name. Expected a string received: ${!filename ? typeof filename : filename.constructor.name}`,
      );
    }
  }
type ParsedResponse = {
    [x: string]: any;
    body?: Response;
}
/**
 * see https://developer.mozilla.org/en-US/docs/Web/HTTP/Basics_of_HTTP/MIME_types/Common_types
 */
const mediaExtensionyTypes = new Map([
    ['jpg', 'image/jpeg']
])

const headerCaseChange = (sentence: string) => sentence.replace(/\-\w/g, (word) => word.toUpperCase().slice(1))
const getFileName = ({ contentDisposition }: Record<string, string>) => contentDisposition.split(";").reduce((accum: Record<string, string|null>, current: string)=> {
    const [key, val] = current.split("=")
    const value = JSON.parse(val?.trim()||"null")
    return {...accum, [key.trim()]: value }
}, {})
const prepareFetch = async () => new Request(new URL('https://picsum.photos/536/354'),{ mode: 'cors' })
const getHeaders = (headers: Headers) => [...headers.entries()].reduce((accum, [key, value]: string[]) => ({...accum, [headerCaseChange(key)]: value}), {})

const processJpegImage = async ({ value }: ReadableStreamReadResult<Uint8Array>) => {
    if (!value?.byteLength) return 'none'
    const [ a, b, c] = value
    if (a === 0xff && b === 0xd8 && c === 0xff) return 'jpg' 
}
const fetchImage = async (): Promise<ParsedResponse> => {
    const request = await prepareFetch()
    const {headers, body} = await fetch(request)
    const reader = body?.getReader()
    const data = await reader?.read()
    if (typeof data === 'undefined') return {}
    const extension = await processJpegImage(data)
    return { extension, headers: getHeaders(headers), data}
}
const saveResponse = async () => {
    const { headers, data, extension } = await fetchImage()
    const mediaType = mediaExtensionyTypes.get(extension) ?? 'application/octet-stream'
    const { filename } = getFileName(headers)
    if (!filename) return new FileNameError(filename)
    const stream = createWriteStream(filename)
    stream.write(data.value)
    return { extension, mediaType, filename, }
}
Enter fullscreen mode Exit fullscreen mode

Inspiration was for this post came from this blog post. Saving and Image...

Resources:

OWASP File Upload Cheatsheet

💖 💪 🙅 🚩
wearypossum4770
Stephen Smith

Posted on May 2, 2024

Join Our Newsletter. No Spam, Only the good stuff.

Sign up to receive the latest update from our blog.

Related