@vitejs/plugin-react-refresh was migrated into @vitejs/plugin-react, and many Emotion + Vite tutorials and boilerplates out there became outdated as a result.
The Vite monorepo has an example react-emotion setup, which seems to works quite well. Here's the gist:

1. Install Emotion Dependencies

Make sure you have the following installed:

yarn add @emotion/react
yarn add -D @emotion/babel-plugin

2. Update vite.config.js

The @vitejs/plugin-react plugin accepts a custom babel config via the babel option. Here, we add the @emotion/babel-plugin plugin we just installed.
Also, to be able to use the css prop in our JSX, we need to instruct @vitejs/plugin-react to use Emotion's jsx function instead of the default jsx-runtime when compiling JSX. We do this by setting the jsxImportSource option to @emotion/react.

import { defineConfig } from "vite";
import react from "@vitejs/plugin-react";

// https://vitejs.dev/config/
export default defineConfig({
  plugins: [
    react({
      jsxImportSource: "@emotion/react",
      babel: {
        plugins: ["@emotion/babel-plugin"],
      },
    }),
  ],
});

Optional: TypeScript support

When using Emotion with TypeScript, your editor may complain that css is not a valid prop. This is because by default, css is not a recognised prop in React and your TypeScript compiler doesn't know about Emotion. Fix this by instructing TypeScript to use types from Emotion instead:

{
  "compilerOptions": {
    "jsxImportSource": "@emotion/react"
  }
}

Hope this helps!