The company owners said the containers in Santos were full of acid and a disinfectant which came into contact with rainwater, causing a reaction.
The area's mayor said at least 66 people were taken to hospital with breathing difficulties.
Officials said the fire had been controlled but that there is still smoke in the area.
The cargo terminal and nearby homes were evacuated and residents were asked to stay inside.
The container terminal was operated by Localfrio, a logistics company, in Guaruja, an area on the eastern side of Santos, in Sao Paulo state.
Santos, a coastal city in southern Brazil's São Paulo state, is a major seaport stretching from the mainland to São Vicente Island. The island's urban center borders the Bay of Santos, known for its wide beach and adjacent 5.3km-long garden. The Museu do Café, occupying a 1922 commodities exchange preserved in the historic district, explores the area's key role in Brazil's coffee trade.
A spokeswoman for the company, which exports chemicals used for refrigeration and general cargo, said the containers were filled with chloric acid and sodium dichloroisocyanurate - a cleaning and disinfectant agent.
Firefighters said rainwater had seeped into the containers causing a chemical reaction.
Local Mayor Mario Antonieta de Brito asked people to stay out of the rain which could "contain chemical elements that can burn the skin".
This is the latest in a series of recent incidents involving chemical companies in Brazil:
Not too long ago, the country's worst environmental disaster, a dam used to hold waste water from an iron mine collapsed in Minas Gerais state in November 2015, killing at least 13 people. Mud containing toxic substances like mercury, arsenic, chromium and manganese reached the Atlantic Ocean
And, shortly after that in April 2015, a fire at a fuel storage facility near Santos raged for nine days, sending a thick black smoke into the sky.
A dam at a separate iron ore mine in Minas Gerais state collapsed in September 2014 causing a landslide that killed three worker... leaving everyone wondering what might be next.