Locate a 360° ceiling camera and what it sees from one fisheye photo

Start from the drugstore model (free sign-up), or draw your own shop. A 360° camera hanging from the ceiling looks straight down through a fisheye lens and takes in the whole shop around it. In its picture, straight shelves and walls come out strongly curved, so it cannot be solved as an ordinary camera. In this article your AI works out, from one still taken by a 360° camera, where the camera is and which part of the floor it covers. It follows the article on a wide-angle security camera and uses the same tools an AI calls over MCP. Every figure and picture below comes from those tools.

1. What was done

• The model is a 36 × 24 m drugstore. A 360° camera hangs from the ceiling over the front aisle by the entrance, 3.3 m up, looking straight down. Its lens is an equidistant fisheye covering 180° across the picture's height, and the picture is 2048 × 2048 pixels (how much the lens bends is assumed).
• The AI reads points of the room itself that show in the picture (corners at floor and ceiling level, and the frames of the entrance, the windows and the back doors) and pairs each with the same point in the model: 15 of the same 40 candidates as in the previous article, the ones that are in the picture and not behind a shelf or a wall. The ceiling's corners are above the camera, but a fisheye still shows them.
• The photo here is made from the model. For each point, the AI read the pixel where the actual camera sees it, with a reading error of ±2 pixels added.
• With solve_photo_camera the AI solved the camera three ways: as an ordinary (pinhole) camera, with a wide-angle lens's distortion (lens: radial) and as a fisheye (lens: fisheye). It placed the fisheye solution as a camera part and compared the floor it covers with the actual camera's, using get_visibility. The back doors are closed, as in business hours.

2. The cameras solved

• Neither an ordinary camera nor a wide-angle lens fits. An ordinary camera cannot take in the 90° from the middle of its picture to the edge. The tool returned the camera that came nearest to the points, one 121.37 m away with a 15.82° field, and the points missed by 200.3 px on average. The wide-angle distortion did not fit either, so the ordinary camera stayed. The tool warned that the position was loosely pinned, that the points missed, and that the lens may be a fisheye.
• With lens: fisheye it fits. The 15 points miss by 1.5 px on average, the field across the picture's height comes out at 178.76° (actually 180°) and the lens's bend coefficient k1 at −0.024 (actually −0.02). The camera solved as a fisheye is 0.02 m from the actual one and agrees on 99.9% of the 221 m² of floor the actual camera sees.
• Its camera part looks straight down, all around. For a fisheye looking straight down, the tool creates a camera part that looks straight down with a 360° horizontal field, seeing out to 90° from straight down. The part covers only what the photo shows.

3. How much of the shop a 360° camera sees

• On the floor, the far ends of the aisles between the shelves are hidden. The farther a point is from straight below the camera, the sooner the line to the floor meets a shelf (1.8 m high). The camera sees the front aisle and the parts of the aisles near it.
• At head height, it covers twice the area of the corner wide-angle camera. Looking down from the ceiling, a line to a point 1.6 m up clears the shelves more easily and reaches far down the aisles.
• The areas are what get_visibility counts on a grid of points 25 cm apart.

4. How to use it, and what it does not know

• Reading the points. Lines curve in a fisheye picture, but the room's corners and the door frames can still be read point by point. Read points both near the middle and near the edges (with 8 or more, the tool also fits how much the lens bends).
• Kinds of fisheye. The tool fits an equidistant fisheye with two terms for its bend. Fisheyes with other projections have not been checked.
• A round picture. A 360° camera's picture is often a round image with black around it. Read the points inside the round image.
• Coverage is geometry only. The part's range is assumed (30 m); whether faces or goods can be told apart depends on the camera's resolution and the distance. With the doors open, the camera may see into the back rooms and the toilets.
• Use a photo with nobody in it. An image in which a face identifies a person is personal information (Camera Image Utilisation Guidebook ver3.0 (in Japanese)). Use a still taken when nobody is there, before opening time for example.
• The photo in this article is made from the model; points read from a real photo may have larger errors.

Tools used

• solve_photo_camera: solve the camera from the points in the photo, as a fisheye, and return the input for a camera part
• edit_space: place the solved camera as a part
• get_visibility: the floor covered, and what is covered at head height
• get_view_image: the plans