Skyplanr
Language

A drone mapping job, requirement by requirement: the plan, the ODM report and what is unverified

A photogrammetry professional asked us to show a job the way a client would check it: each requirement, what was delivered, the evidence and a status. The planned GSD sits apart as an assumption, and everything nobody measured is marked UNVERIFIED.

By Sander Helgesen · Last checked

In short

  • Of seven requirements, two are met (coordinate system, formats), two are not (a DSM of the whole area, full coverage) and three are UNVERIFIED: the 10 cm accuracy, and whether buildings, paths and parking rows, and tree crowns, are recognisable, which only we have judged.
  • Without independent checkpoints, accuracy is relative only. The report's relative CE90 of 0.119 m is ODM's estimate inside the model, not a measurement against surveyed points.
  • Planned GSD is not verified accuracy. It sets a limit on the smallest detail the photos can show, nothing more.
  • LoD is geometric detail, not a pixel size; the LoD vs GSD guide explains the difference.
  • The plan is Skyplanr's for this area; the photos are OpenDroneMap's public zoo set from 2016 and were not flown from this plan.

Where each part comes from

We can't publish a client's job, and the test account has no flight of its own over a mapping site, so the example puts together public pieces and says which is which.

  • The brief: written for this example after the processing had run, in features, accuracy, format and coverage, with no GSD in it.
  • The plan: Skyplanr planned the brief for a DJI Air 3S on the test account, on a polygon we drew around the photos' footprint at the Cleveland Metroparks Zoo in Ohio. The polygon follows the dataset, not the zoo's boundary.
  • The photos: OpenDroneMap's public zoo dataset (CC0), taken on 2016-10-25 with a Sony DSC-WX220 on an eBee fixed-wing drone, which wrote the GPS positions. We processed the first 200 of 524 by file name, the set we use to test processing releases; the other 324 would likely fill the edges.
  • The results: Skyplanr's processing (ODM 3.6.2, worker 1.0.9, job cbec3f46) on those photos. report.json and the ODM report PDF are unchanged; figures 1 to 5 are crops of its outputs (figure 1 the job's preview, 2 the full orthophoto, 3 to 5 the report PDF).

Requirements versus results

The brief: an orthophoto and surface model of the zoo grounds for a site plan. Buildings, paths, parking rows and tree crowns must be recognisable. 10 cm horizontal accuracy. A GeoTIFF orthophoto and DSM in UTM zone 17N and a LAZ point cloud. Coverage: the whole polygon.

The requirements in the brief, what the job delivered, the evidence and the status. Met and Not met rest on a tool output in checks.txt or a defect visible in the published report; UNVERIFIED means it rests only on our own judgement, or nothing was measured. The figures follow the table, our checks are in checks.txt.

  • Buildings, paths and parking rows recognisable

    Delivered
    Orthophoto at 3.26 cm/px. All three are sharp in the full-resolution crop. Judged by eye by us, not independently; the full file is not public.
    Evidence
    Figures 1 and 2
    Status
    UNVERIFIED

  • Tree crowns recognisable

    Delivered
    Visible as crowns, but smeared where photos are stitched. Whether that is good enough is the client's call.
    Evidence
    Figure 2
    Status
    UNVERIFIED

  • Surface model (DSM) of the whole area

    Delivered
    DSM at 6.25 cm/px from 32,234,160 dense points. Usable over most of the area, but flat discs and steps along the wooded south edge and in places along the north-east edge.
    Evidence
    Figure 3 (ODM report p. 2)
    Status
    Not met

  • 10 cm horizontal accuracy (no measure or checkpoint count named)

    Delivered
    No GCPs, RTK or checkpoints; positions are the photos' GPS. ODM's estimate inside the model: relative CE90 0.119 m. The solved camera positions differ from the GPS by 2.80 m east and 3.84 m north (RMS). Nothing is measured against the ground.
    Evidence
    Figure 5 (ODM report p. 4); report.json: accuracy.class relative, checkpoint_rms_m null
    Status
    UNVERIFIED

  • UTM zone 17N (no vertical datum named)

    Delivered
    Rasters tagged EPSG:32617. The LAZ carries the same projection as WKT, without an EPSG code. Vertical datum unknown (report.json: crs.vertical unknown).
    Evidence
    checks.txt §1 and 2
    Status
    Met

  • GeoTIFF orthophoto and DSM, LAZ point cloud

    Delivered
    ortho.tif and dsm.tif (Cloud Optimized GeoTIFF), dtm.tif, pointcloud.copc.laz (LAZ 1.4 with COPC), ODM's report.pdf and report.json.
    Evidence
    checks.txt §1 and 2
    Status
    Met

  • Coverage of the whole plan polygon (59.7 ha, drawn around the dataset's footprint)

    Delivered
    The orthophoto covers 97.3 % of the polygon (the report: 92.5 % of the upload's own area). Holes along the edges; figure 4 shows thin coverage over water and dense trees.
    Evidence
    Figure 4; checks.txt §3
    Status
    Not met
Orthophoto of the Cleveland zoo seen straight down: buildings, paths, parking lots, a pond and trees in autumn colours.
Figure 1. The orthophoto, the job's preview (1024 px wide). The full file is 3.26 cm/px.
Full-resolution crop: parked cars in rows, a rusty roof and tree crowns smeared along a seam.
Figure 2. Full-resolution crop (1600 × 1000 pixels, about 52 × 33 m). Cars, roof sheets and kerbs are sharp; the tree crowns on the right smear where two photos meet.
Surface model in a colour scale from 122 to 209 metres, with flat discs and steps along the south edge.
Figure 3. The DSM from the ODM report, page 2. The flat discs along the south edge are interpolation where the point cloud has no points.
Map of how many photos see each point: green over most of the site, red and white along the edges and over the pond.
Figure 4. Number of photos that see each point (ODM report, page 4). White holes and red (2 photos) are where coverage is weak.
Table from the ODM report with GPS errors, 3D errors and absolute and relative CE90 and LE90.
Figure 5. The error tables in the ODM report, page 4. What ODM calls absolute is the distance between the photos' GPS and the solved camera positions, not a measurement against the ground.

The plan: assumptions, not results

The brief names no GSD, so Skyplanr picked about 2.8 cm/px from the features. It is a planning input, so it is not in the results table, and the processed photos came from another camera at another height anyway. The plan's summary does say that 2.8 cm/px "still supports 10 cm accuracy once you add surveyed ground control points". That is a rule of thumb; only checkpoints would show it.

Two more things in the plan matter for the results. It keeps the drone within 500 m of the pilot's takeoff, so one mission covers only 19.0 of the 59.7 ha and the rest needs two more takeoffs. And for the 10 cm target it asks for 3 checkpoints (its CSV export suggests 2), well short of the 20 the NSSDA asks for. Two or three show whether something is badly wrong; they are not enough to document accuracy.

Skyplanr's plan for the same area, next to the photos that were actually processed. Every plan value is an assumption; the photos were not taken from this plan.

  • Drone and camera

    Plan (assumption)
    DJI Air 3S, 12 MP mode
    Photos processed
    Sony DSC-WX220 on an eBee fixed-wing mapping drone (EXIF), 2016

  • GSD

    Plan (assumption)
    About 2.8 cm/px at 80 m above the ground with terrain follow (75 to 114 m above takeoff)
    Photos processed
    3.62 cm/px in the reconstruction, ortho 3.26 cm/px (measured by ODM)

  • Overlap

    Plan (assumption)
    85 % front and 81 % side on the ground (80/75 at the treetops)
    Photos processed
    Not given as front/side; most of the area is seen by 5 or more photos (figure 4)

  • Area

    Plan (assumption)
    19.0 of 59.7 ha in this mission. The west part is over 500 m from the takeoff and saved as two missions of their own (23.9 and 20.3 ha) that overlap this one's edge.
    Photos processed
    97.3 % of the polygon in the orthophoto

  • Photos

    Plan (assumption)
    845 (811 straight down, 34 oblique around the tallest building in this mission's area) on 5 batteries, about 66 min
    Photos processed
    200 (the first 200 of 524), 199 placed

  • Plan check

    Plan (assumption)
    0 errors, 11 warnings, among them: the straight-down photos cover 39 % of the polygon (their footprints reach past the mission's lines), parts of the polygon are beyond line of sight from the takeoff, building and tree heights are estimates
    Photos processed
    No plan

  • GCPs and checkpoints

    Plan (assumption)
    The plan's rationale asks for 10 to 15 GCPs and 3 checkpoints, measured with RTK in UTM 17N; the Ground control (CSV) export suggests 10 and 2, with targets of at least 29 cm
    Photos processed
    None
The plan in the app: a dense grid of waypoints over the east part of the zoo, and the west part hatched as not covered.
Figure 6. The plan in Skyplanr (the test account): 811 straight-down photos over the east part, the rest hatched because it is over 500 m from the takeoff. Map and imagery: Mapbox, OpenStreetMap, Maxar.

How the accuracy would be checked

This is what turns the UNVERIFIED accuracy row into met or not met. None of it was done for this example.

  1. Pin the requirement down

    "10 cm horizontal" needs a measure. The NSSDA reports accuracy at 95 % confidence; ASPRS Edition 2 reports RMSE. Agree which one, and the coordinate system and height datum, before the flight.

  2. Survey checkpoints and keep them out of the processing

    Measure them with RTK GNSS or a total station, on well-defined points (paint marks, kerb corners, targets) spread over the whole area, edges included. The NSSDA asks for at least 20; ASPRS Edition 2 asks for at least 30 for a fully compliant assessment. GCPs, if you use them, are separate points.

  3. Measure each checkpoint in the delivered files

    Read each checkpoint's coordinates in the finished orthophoto and DSM, not in the processing software's preview, and compute the differences in easting, northing and height.

  4. Report the number with its method

    Give RMSE per axis (or accuracy at 95 % confidence), the number of checkpoints, the instrument, and any point you dropped and why. Then set the status: met or not met.

  5. Expect this job to fail it

    Without GCPs or RTK the position rests on the photos' GPS, and here the solved camera positions differ from it by 2.80 m east and 3.84 m north (RMS). A 10 cm target needs ground control or an RTK drone first.

What the report's numbers do and do not say

  • GSD 3.62 cm/px, ortho 3.26 cm/px: resolution measured by ODM, not accuracy.
  • Reprojection error 0.837 px: the photos agree with each other. The model is consistent; that does not put it in the right place.
  • Relative CE90 0.119 m, LE90 0.213 m: ODM's estimate, from the reprojection error, of how far points could move inside the model. Useful for comparing jobs. The "3D" RMS in figure 5 (0.068 / 0.082 / 0.190 m in X, Y and Z) comes from the same estimate, so the 7 to 8 cm there is no evidence that a 10 cm target is met.
  • overlap_median 2 in report.json: the median number of photos per sparse tie point, not photos per area. Figure 4 counts photos per point of the surface.
  • "Absolute" CE90 1.485 m, LE90 1.83 m, and GPS RMS 2.802 / 3.841 m: the distance between the photos' GPS and the solved camera positions. CE90 is the 90th percentile; the RMS is pulled up by a few photos far off. Neither tests the model against the ground.
  • Coverage 92.5 %: the orthophoto's share of the area set on the upload. Against the plan's polygon we measured 97.3 % with the same method.
  • LoD, if a brief names one, is the geometric detail of a 3D model. It has no pixel size and no accuracy; ask for the features and the accuracy separately.

Where Skyplanr fits in

Skyplanr plans the flight for DJI Fly, labels its GSD as an assumption, suggests GCP and checkpoint positions with the targets' size, and checks the plan before you get the KMZ. Its processing builds the orthophoto, DSM, point cloud and the ODM report, and labels every result relative accuracy, because it does not measure checkpoints yet. The mapping guide covers the flight side.

See the technology

Questions

Does this job meet 10 cm horizontal accuracy?

Unknown, and probably not. Nobody measured checkpoints, so the row is UNVERIFIED. Without ground control or RTK the position rests on the photos' GPS tags, which scatter by metres around the solved camera positions; the model's absolute offset is unknown.

Were these photos flown from the Skyplanr plan?

No. They are OpenDroneMap's public zoo dataset from 2016, taken with a Sony compact camera on a fixed-wing drone. Skyplanr planned the same area for a DJI Air 3S so the plan and the results can sit side by side.

Can Skyplanr check accuracy with checkpoints?

Not yet. Skyplanr suggests checkpoint positions in the plan and exports them for an RTK rover, but the processing does not read surveyed coordinates, so every result is labelled relative accuracy.

Sources

Skyplanr is not affiliated with the third-party guides linked here.

  1. OpenDroneMap: odm_data_zoo, the public test photos used here (CC0 1.0) (github.com)
  2. FGDC: National Standard for Spatial Data Accuracy (FGDC-STD-007.3-1998), 20 checkpoints, 95 % confidence (fgdc.gov)
  3. LIDAR Magazine: The ASPRS Positional Accuracy Standards, Edition 2 (30 checkpoints, RMSE) (lidarmag.com)
  4. Pix4D community: check points versus GCPs (check points are only for accuracy reporting) (community.pix4d.com)
  5. OpenDroneMap documentation: outputs and the quality report (docs.opendronemap.org)
  6. COPC: Cloud Optimized Point Cloud specification (LAZ 1.4) (copc.io)
  7. DJI Developer: DJI WPML overview (the KMZ format DJI Fly reads, the plan's export) (developer.dji.com)

This guide describes how DJI Fly behaved when it was last checked; DJI can change the app at any time. It is advice, not a permission: you are the pilot in command, and the site, the geo-zones and the rules are yours to check before take-off.

Let Skyplanr write the file.

Describe the job, and Skyplanr plans the waypoint mission, checks it and gives you the KMZ for DJI Fly.

Get started freeSee pricing