In addition to systems for sensing, decision making, and communicating, self-driving cars need reliable maps. CARMERA Inc. has been providing “living maps” and navigational data for autonomous vehicles.

The New York-based company has about 60 employees and has been conducting trials with partners in the Brooklyn Navy Yard. Ethan Sorrelgreen, head of product at CARMERA, recently spoke with The Robot Report about challenges around creating reliable maps at scale.

Going out to get map data

The first challenge is gathering enough data. CARMERA works with partner fleets to create high-definition maps that it says can “regenerate in real time.”

“We take a two-pronged approach to mapping,” Sorreelgreen said. “We use lidar, imaging, GPS, and IMUs [inertial measurement units] in the car to generate HD maps for localization and to improve safety.”

“We’re also leveraging human-driven delivery vehicles and telematics to track them, plus video monitoring,” he explained. “With the capture of HD video, we have the data to maintain maps and to understand what’s changing.”

Striking the right balance between processing petabytes of video or using too little to scale for reducing “noisy” signals is another data management challenge.

Last month, CARMERA and Toyota Research Institute-Advanced Development (TRI-AD) used consumer-grade cameras to map roads in Tokyo and Michigan.

“Our work together demonstrates the power of developing systems that are built to take advantage of the ubiquity — and understand the constraints — of automotive technology widely in use today,” stated Ro Gupta, CEO of CARMERA. “It shows that with the right approach, companies like TRI-AD can start building the data necessary for automated driving, without having to rely on expensive or proprietary third-party hardware.”

Fleet size and data management

Service-level agreements typically specify what percentage of road segments or which ones much be refreshed and how often, Sorrelgreen said.

“With the right-size fleets, CARMERA can meet data SLAs for our customers, providing operational intelligence,” he added. “For instance, in Seattle, we make an effort to cover all major highways and access roads on a weekly basis. It’s not all done with fleet vehicles; there is some filling in with our own technicians.”

“They keep the lenses clean and see if Wi-Fi hotspots are working,” said Sorrelgreen. “With a small fleet, we get 99% of highways in Tacoma, Everett, and Easton, [Wash.] There is an optimal size for that fleet, which is proprietary.”

“We haven’t needed more technicians in New York so far, but we have noticed that there is a difference in road types to fill in; vehicles can drive faster on highways,” he said. “Downtown San Francisco is worse than Manhattan in terms of getting Pac-Man-style coverage of streets.”

Is weather a challenge? “Not really,” responded Sorrelgreen. “Some of our testing for maintaining data was at M-City, and Michigan gets much worse weather than we do in Seattle. We’ve actively tested our systems in snow and rain.”

“Our software is flexible, and we put it on fleet vehicles in a ruggedized Android device,” he explained. “We’re working with companies such as Toyota on production hardware.”

CARMERA works with city governments

In addition to helping with management of autonomous fleets, CARMERA has been sharing the “data exhaust” of its HD maps for free with municipal agencies, Sorrelgreen said.

“We started working in New York city way before the Brooklyn Navy Yard deployment,” he added. “We’ve also covered Manhattan, and we’re trying to get to refreshing data on 99% of streets there every 30 days.”

The mapping data could help utility providers, traffic bureaus, and mass-transit providers. “With the data exhaust, we’re exploring the technology in a different way,” said Sorrelgreen. “In the past, it was hard to tell how active construction was impacting buses or pedestrians. We’re working with the Downtown Brooklyn Partnership on providing data on dense infrastructure.”

“We’ve recently worked with the Transit Tech Lab and the MTA [Metropolitan Transportation Authority] in New York City,” he said. “We hope to learn how public-transit agencies will value our data. It would be interesting if buses could be routed to go faster and if drivers could be given more enforcement instructions. More people would ride them.”

CARMERA calibrates sensors

As part of its effort to map various cities, CARMERA must choose which vehicles to equip with its sensors. For instance, CARMERA has mounted its sensors on delivery fleets and its own technicians’ cars, but it has not yet put them on mass-transit vehicles, Sorrelgreen said.

“There is specific integration with any vehicle we want to use,” he noted. “When looking at a partnership with with an automaker, we have to calibrate computer-vision models for the sensors they use.”

“The hardware on R&D autonomous vehicles is the same as on some production vehicles today,” said Sorrelgreen. “We could calibrate our systems to work with them, and we would protect privacy while gathering data.”