Documentation

Tolerance and tessellation

How one chord-height tolerance decides segment counts, display decimals and curve tessellation across the CAD kernels.

One tolerance, every curve

A tolerance is the largest distance the model may be from the true shape, in millimetres. For a circle drawn as a polygon that distance is the chord's sagitta r(1 − cos(π/n)), so the number of segments follows from the tolerance. packages/core/src/cad/tolerance.ts (39 lines) holds that rule; the editor's renderer, the assistant's element catalogue, the entity tessellator and the tests all use it.

Ø20 circle at three tolerances Drag to orbit · scroll to zoom · right-drag to pan

The same radius-10 circle as segmentsFor gives it: 10 segments at 0.5 mm (chord error 0.489), 23 at 0.1 mm (0.093) and 71 at the 0.01 mm default (0.0098). Each polygon is extruded 2 mm.

Constants

Name	Value	Meaning
TOLERANCE_PRESETS	0.1, 0.05, 0.01, 0.005, 0.002, 0.001, 0.0005	what the UI offers, coarse to fine
DEFAULT_TOLERANCE	0.01	editor and tessellation default
MIN_TOLERANCE	0.0001	below this isTolerance is false
MAX_TOLERANCE	10	above this isTolerance is false
MIN_CURVE_SEGMENTS	8	fewest segments of a circle
MAX_CURVE_SEGMENTS	2048	default cap of segmentsFor
MAX_SEGMENTS (tessellate.ts)	65 536	hard cap per entity in the entity tessellator
MIN_TOLERANCE (tessellate.ts)	1e-6	floor so a zero tolerance cannot produce unbounded counts

tolerance.ts

export const isTolerance: (value: unknown) => value is number      // finite, in [0.0001, 10]
/** Smallest n ≥ 8 with r(1 − cos(π/n)) ≤ tolerance, capped. */
export const segmentsFor: (radius: number, tolerance: number, cap?: number /* 2048 */) => number
export const chordError: (radius: number, segments: number) => number   // r(1 − cos(π/n))
/** 0.001 → 3, 0.05 → 2, 0.5 → 1; at most 6. */
export const decimalsForTolerance: (tolerance: number) => number

Presets in numbers

Real segmentsFor, chordError and decimalsForTolerance output for each preset:

Tolerance, mm	segments r 10	chord error r 10	decimals	segments r 50
0.1	23	9.314e-2	1	50
0.05	32	4.815e-2	2	71
0.01 (default)	71	9.788e-3	2	158
0.005	100	4.934e-3	3	223
0.002	158	1.977e-3	3	352
0.001	223	9.923e-4	3	497
0.0005	315	4.973e-4	4	703

Edge cases, also real: segmentsFor(1000, 0.0001) = 2048 (the cap), segmentsFor(2, 5) = 8 (tolerance ≥ radius), isTolerance(0.00005) = false, isTolerance(10) = true, decimalsForTolerance(0.0001) = 4.

Tessellating drawing entities

tessellate(entity, tolerance) in tessellate.ts turns any drawing entity into polylines within the chord tolerance. That polyline is what the CAM layer offsets and what the viewport renders. A closed shape returns a ring whose first and last points differ; the closing segment is implicit.

Entity	Rule
Line	its two end points
Circle, Arc	arc segment count from radius, sweep and tolerance, capped at 65 536
Ellipse	uniform parameter step h = √(8·tol / a), a the larger radius, at least 8 segments
Spline	analytic bound of splinePolyline
Polyline	bulged segments as arcs
Rectangle	four corners
Text, Dimension	no outline: empty list

For an ellipse with radii 20 and 10, ellipseSegmentCount gives 100 segments at 0.01 mm and 32 at 0.1 mm.

Where tolerances meet

Kernel	Tolerance parameter	Default
Sketch profiles	sketchProfiles(sketch, tolerance)	0.01 mm
Mesh primitives	meshOf(solid, tolerance)	0.1 mm
CSG coincidence	CsgOptions.tolerance	1e-7 × extent
B-rep tessellation	TessellateOptions.chordTol	diagonal / 2000, in [0.001, 0.05]
B-rep Boolean coincidence	BooleanOptions.tol	1e-8 × size, ≥ 1e-7
Helix chords	stepsPerTurn	0.01 mm

Display tolerance is not machining tolerance

The drawing tolerance controls how curves are approximated. CAM verification and G-code arc fitting carry their own tolerances; see the CAM kernel.

Troubleshooting

Symptom	Cause	Fix
circles look polygonal	tolerance 0.1 on a large radius	choose a finer preset; segments grow with √r
viewport slow on a large drawing	very fine tolerance on many curves	0.01 is enough for display; keep finer values for export
length shows too many decimals	decimals follow the tolerance	decimalsForTolerance(0.01) = 2
a value is rejected as tolerance	outside [0.0001, 10]	use a preset