The fire, which started burning on Saturday, had doubled in size to 10,000 acres from Monday night into Tuesday morning.
About 42 more were expected to arrive at Wahtum Lake by early Sunday afternoon.
The fire spread 13 miles in 16 hours, pushed by winds overnight that ranged from 30 mph to 40 mph in tinder-dry conditions, Brown said.
The fire is burning near the hikers in the Eagle Creek Area just outside of Portland.
Oregon Fire Marshal incident commander Ian Yocum said firefighters lost one residential structure and 4 outbuildings Tuesday, but crews managed to save countless others.
Interstate 84 was closed Monday due to poor visibility from smoke and flames close to the road.
Level 3 (GO NOW) evacuations were put in place for the communities of Dodson and Warrendale. Residents as far west as Hillsboro reported ash falling.
Oregon State Police said the fire appears to have been started by fireworks, and a suspect has been identified.
Putin warns against "military hysteria"
The United States is calling for the strongest possible sanctions and wants the 15-member council to hold a vote next week. Vassily Nebenzia said Russian Federation wants a reference to the need for political dialogue based on recent initiatives.
"If an ember or spark came and landed on one of the trails, (park staff) wouldn't want to have to worry about evacuating people hiking", said Toni Droscher, a spokeswoman for state parks. Level 1 evacuations means people should be aware of fire danger and ready for potential evacuations. The additional bad news is that unconfirmed reports have the fire jumping the Columbia River and sparking a new fire on the Washington state side.
The Multnomah County Sheriff's Office says fire onlookers are hindering rescue operations. State Route 14 through the Washington side of the Gorge is closed to semi-truck traffic.
Families reunite after campers were evacuated from a forest fire in the Eagle Creek area of the Columbia River Gorge Sunday, Sept. 3, 2017.
The investigation is being assisted by the U.S. Forest Service, the Hood River district attorney's office, the U.S. attorney's office and the Hood River Juvenile Department.
Mountain Wave Search and Rescue president Russ Gubele says about 104 people have made their way down a different trail to safety.
The first group made it out by about 10:30 a.m. and the last group by about 1 or 1:30 p.m.
The hikers were forced to spend the night outside near Tunnel Falls, about six miles from the trailhead, and were still waiting to be rescued Sunday morning, Oregon Live reported.