Next.js Discord

Discord Forum

Help with env variables using docker-compose pls

Unanswered
Havana posted this in #help-forum
Open in Discord
HavanaOP
This is my file tree:
├── challenge
│   ├── bot
│   │   ├── Dockerfile
│   └── web
│       ├── prod.Dockerfile
├── docker-compose.yaml
├── .env
└── README.md

I'm trying to copy over my .env variables without success. This is my docker compose:
version: '3.8'
services:      
  postgres:
    image: 'postgres:latest'
    restart: always
    environment:
      POSTGRES_PASSWORD: ${DB_PASSWORD}
      POSTGRES_USER: ${DB_USER}
      POSTGRES_DB: ${DB_NAME}
      DATABASE_URL: ${DATABASE_URL}
    volumes:
      - 'postgres_data:/var/lib/postgresql/data'
    ports:
      - '${DB_PORT}:${DB_PORT}'
    networks:
      - main_network
  bot:
    restart: always
    depends_on:
      - postgres
    build:
      context: challenge/bot
      dockerfile: Dockerfile
    volumes:
      - ./challenge/bot:/home/pptruser/app
      - ./challenge/bot/node_modules:/home/pptruser/app/node_modules
    ports:
      - '${XSSBOT_PORT}:${XSSBOT_PORT}'
    networks:
      - main_network
    environment:
      XSSBOT_PORT: ${XSSBOT_PORT}
      XSSBOT_HOST: ${XSSBOT_HOST}
  web:
    restart: always
    build:
      context: challenge/web
      dockerfile: prod.Dockerfile
    ports:
      - '3000:3000'
    depends_on:
      - postgres
    volumes:
      - ./challenge/web:/app
      - ./challenge/web/node_modules:/app/node_modules
      - ./challenge/web/.next:/app/.next
    networks:
      - main_network
    environment:
      #change this one to the actual final url (e.g https://ecorp.ctf.intigriti.io/api/graphql )
      - NEXT_PUBLIC_GRAPHQL_ENDPOINT_URL='http://localhost:3000/api/graphql'
      # Don't change these here
      - DATABASE_URL=${DATABASE_URL}
      - DB_PASSWORD=${DB_PASSWORD} 
      - XSSBOT_PORT=${XSSBOT_PORT}
      - XSSBOT_HOST=${XSSBOT_HOST}
      - JWT_SECRET=${JWT_SECRET}
      - ALLOWED_USER_AGENT_ID=${ALLOWED_USER_AGENT_ID}
      - EXAMPLE_ID=${EXAMPLE_ID}
      - FLAG=${FLAG}
      - ALLOWED_USER_AGENT_REGEX=${ALLOWED_USER_AGENT_REGEX}
...

84 Replies

HavanaOP
the postgres service works without issues. The env variables all work. Same with the bot.
Only the nextjs app gives me this error:
 PrismaClientConstructorValidationError: Invalid value undefined for datasource "db" provided to PrismaClient constructor.
32.49 It should have this form: { url: "CONNECTION_STRING" }

referencing this:
console.log("#################  ENV: ", process.env)
export const prisma = globalForPrisma.prisma || new PrismaClient({
    log: ["query", "info", "warn", "error"],
    datasources: {
        db: {
            url: process.env.DATABASE_URL,
        }
    }
})

in my challenge/web/lib/prisma.ts file
the console log doesn't show any of my variables
i also tried specifying my env_file: like this:
  web:
    restart: always
    env_file:
      - ./.env
still nothing
with the way next works during build expecting an actual env file, i still think you are gonna need a .env in project root
HavanaOP
dammit... Yeeah but unfortunately this is going to have to be used with kubernetes and they told me i can't use a makefile to copy over the .env file to ./challenge/web/.env
and i can't COPY from the prod.Dockerfile because the context is set to the web folder and not the root
why are you using compose with k8s?!?!
HavanaOP
idfk lmfao i'm just a noob front-end dev
you are trying to solve a problem that wont exist
HavanaOP
it's been a shitshow
compose is for local baby stuff. k8s is defined very differently, referenced differently, etc
HavanaOP
yeah but i always had issues with nextjs env, i already had made a post some time ago about a similar issue... I just don't get it why i can't use my dockercompose envs
besides i'm not running it with k8 right now, i'm just running locally so far
HavanaOP
sheeesh then what do i do? Do i just tell them that they gotta deal with it and manually copy the .env file?
injecting env vars from the host directly is a shitshow you do NOT want to debug. trust me
it may not seem like it now, but using a file is a godsend
HavanaOP
i mean sure, i don't like using compose for envs anyway, but i don't know how else to do it
compose env is == to host env btw
since its just local, just copy your .env to the app dir
it will be done differently in k8s anyway
HavanaOP
i can do it locally but what do i tell them to do? To copy the example env file to the web folder manually?
they're not experienced with k8 either
this is the first time for both parties lol
🥶 haha.
so 2 things
1. they should be building in a pipeline before deploying. not inside k8s on init
2. the pipeline should pull the secrets from a secret manager and build the .env file for the build context
HavanaOP
i got an azure-pipeline.yml and deployment.yml files
that they gaave me as an example of what they're doing
but i literally couldn't understand anything from it
dont take this as a posh statement, just take it at face value. docker is already confusing for people who have never done devops. you add k8s in without docker knowledge and it can melt your brain a bit
so its ok that it looks like gibberish
HavanaOP
you are diving head first into the deep end
i cant read that on mobile :/
need it as a pastebin or something
HavanaOP
that's the azure-pipeline
and this is the prod deployment.yaml
https://pastebin.com/fS2N6teN
i legit tried to watch some courses about k8 and i got bored to death and couldn't understand anything
and i've deal with reverse engineering malware samples, lots of assembly, reversing Java bytecode and patching stuff
yet i've never been so confused in my life
so… they are mixing 2 separate technologies they shouldnt be mixing imo
they are using docker compose in the pipeline which is lazy and error prone
they need to build the containers individually properly
HavanaOP
this is for a CTF Challenge infrastructure btw
so it's going to have many many (purposefully vulnerable) web apps
yeah thats not the point tho
the point is that the pipeline itself is shittily written
which is not your fault
but is the case
HavanaOP
well of course, as i said, it's their first time with k8 too lol
tell them to rewrite without using compose at all
there are tons of pipeline examples for building multiple docker containers
when they do that you will be able to copy the env no problem
HavanaOP
i think they're gonna laugh in my face cause they're only having issues with my web app so far 😂
HavanaOP
so basically it should be their job to deal with .env files? Even for the nextjs app? It's still confusing me
is there nothing i can do on my end?
well, besides dealing with those deployment files
which i don't understand in the slightest
@Havana so basically it should be their job to deal with .env files? Even for the nextjs app? It's still confusing me
you either have a copied .env file in the app dir, or you remove the bad compose obfuscation fucking up the copy. those are truely your options
its not your job to add shit on their pile of shit
tell them to clean the shit
😂
(this was literally my job for years)
HavanaOP
LOL alright, i'll try the first one first, then when my patience runs out i'll go with the latter
thank you very much for you help. Even though i'm kind of sad there wasn't a better way to do this on my end and that nextjs is a bit whiny about this env thing
@Havana thank you very much for you help. Even though i'm kind of sad there wasn't a better way to do this on my end and that nextjs is a bit whiny about this env thing
hopefully MORE apps will be like this in the future. the 12 factor app idiom is super insecure and leaks env to the host
which adds multiple attack vectors
HavanaOP
what's the 12 factor app
oh nvm just googled it
its a shitty ideology made a whole back that is taking ages to break people of
HavanaOP
i'm missing a shitton of backend theory stuff apparently
@Havana i'm missing a shitton of backend theory stuff apparently
yeah again its ok. its stuff i learned over YEARS.
its too much for any 1 person in a tiny span of time
HavanaOP
got it, thanks a lot again
European sprat
I pass env vars and args into the dockerfile, I don't copy the .env into it
HavanaOP
You wot
What do you mean
European sprat
At least for me, some variables are needed during build time so I pass them as args during the next build process