zlacker

My gut has the same feeling, but to put numbers on it, for a 1.76 trillion parameter model...

- It takes 10s of millions of dollars in GPU time for training?

- Curation of data to train on

- Maybe 10s of thousands of man hours for reinforcement?

- How many lines of code are written for the nets and data pipelines?

Does anyone have any insight on these numbers?

>>consum+(OP)
100M for GPT-4 I think. They would be in a strong position to negotiate a discount on that GPU bill though.