nextjs multistage docker build behind nginx proxy

I have a multistage docker build for my nextjs frontend app. It looks like this:

# Do the npm install or yarn install in the full image
FROM mhart/alpine-node AS builder
WORKDIR /app
COPY ./package.json ./
RUN npm install
COPY . .
RUN npm run build
RUN npm run export

FROM nginx
EXPOSE 3000
COPY ./nginx/default.conf /etc/nginx/conf.d/default.conf
COPY --from=builder /app/out /usr/share/nginx/html

The problem is now I have dynamic routes. Therefore I cannot run the command next export, which builds my app as a static app. NextJS states clearly that not every app can be deployed as a static app. If your app needs to generate dynamic pages at the runtime, you can't deploy it as a static app. https://nextjs.org/learn/excel/static-html-export

Now the problem is that in my docker deployment I depend on an entry index.html file to be copied from my nextjs exported folder into /usr/share/nginx/html folder in nginx. Is there a way for me to export 3000 from nextjs to nginx, but not build static files and still have it sit behind nginx, what do i need to copy over to nginx if not the index.html file?

The nginx webserver that routes the traffic to either the client or the server is as follows:

upstream client {
  server client:3000;
}

upstream api {
  server api:4000;
}

server {
  listen 80;

  location / {
    proxy_pass http://client;
  }

  location /sockjs-node {
    proxy_pass http://client;
    proxy_http_version 1.1;
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection "Upgrade";
  }

  location /api {
    rewrite /api/(.*) /$1 break;
    proxy_pass http://api;
  }
}