6 Tips for Holding Tight Tolerances


There are few things that a machinist likes more than when they get a print and see this: +/- 0.005”. Holding five thousandths of an inch is child’s play for any good machinist – they might as well mill the part with their eyes closed. But, then there are those jobs that are a bit more demanding. Add another zero, and now you’ve got: 0.0005”. Holding five tenths of a thou is a whole different story. It’s the difference between the thickness of a human hair and a white blood cell. When it comes to holding tight tolerances, here’s a few recommendations that can keep your parts in spec.

Spindle warm up and warming up your CNC machine in all axes helps in holding tight tolerances.
Spindle warm up and a warm up routine can help in holding tight tolerances when machining.

1. Spindle Warm Up for Holding Tight Tolerances

Run a warmup routine – While this is standard procedure with most CNC machines, consider running something a bit more strenuous. A typical procedure will only warm up the spindle, which is critical for spreading grease to prevent premature bearing wear. But, you also need to allow the internal components to reach a steady operating temperature to account for thermal expansion. Now, all of this is fine if you’re only looking to hold tight tolerances in your Z axis, but if you combine the spindle warm up with machine movement in all axis, this will help even further. Allowing the machine to run for 10-20 minutes with all components moving allows for the components to reach an ideal temperature, and will help mitigate the effects of thermal expansion during milling. No matter what, at the end of your warmup, make sure to measure all your tools for absolute precision and holding tight tolerances.

Tool selection can be a factor in holding tight tolerances.
Tool selection can be a factor in holding tight tolerances. Use your roughing tool for the “heavy lifting” so that the finishing tool exhibits less wear and maintains precision.

2. Tool Selection for Holding Tight Tolerances

Choose your tools carefully – When you’re dealing with these unforgiving tolerances, be sure to be accommodating with your tooling. You’ll want to make sure to have specific tools for roughing and finishing, allowing the roughing tool to take the brunt of the wear, while the finishing tool is saved for only the final passes, will ensure a repeatable process for creating accurate parts.

Gauge pins are a handy tool in holding tight tolerances in that you can machine an under sized feature and then dial it in.
Gauge pins can be used measure an under-dimensioned feature before machining it to an exact size.

3. Compensation for Holding Tight Tolerances

Compensate your tools – Tool manufacturers aren’t perfect, so they engineer their tools to be a little forgiving. They know that if you’re going to make something using their tools, you’ll be a lot happier if the feature it cuts comes out under-dimensioned instead of over-dimensioned. Just like a haircut: you can take more off, but you can’t put it back on. Knowing this, you’ll want to make sure the first thing you do when setting up a precise job is to dial in your actual tool diameter. You can do this several ways, but my preferred method is to mill a feature and then use accurate tools to verify the dimension – gage pins or blocks work well for this. It’s easy – if you interpolate a 0.250” hole with a 0.236” tool and only a 0.248” gage pin will fit, then your tool is undersized by 0.001” (use half of the value since it is undersized on each side). You would compensate your size to 0.235” at this point, either through your CAM software or utilizing Tool Comp commands in your cut file.

Temperature sgould be considered if holding tight tolerances is critical in your manufacturing.
Temperature impacts accuracy due to thermal growth. So, be mindful of your environment and machine location.

4. Temperature for Holding Tight Tolerances

Thermally Stabilize – This is one of the most important things on this list for holding tight tolerances because it can make a huge difference and you may not even notice it. Pay attention to where your machine is located. Is it near a window, if so, does the sun shine on it during parts of the day? Does the AC kick on in the afternoon and blow cold air on the machine cabin? Is your material kept a sweltering warehouse, then brought into a chilly 68° environment? These all seem innocent but can create a huge headache in your process. Thermal expansion or contraction of the milling machine or the material you cut can create large variances in your process. Put these all on lockdown – keep your machine and material in a temperature controlled climate, unaffected by sunlight, and you will reap the rewards – consistency in your process.

Ball bar testing and frequent machine calibration help in holding tight tolerances.
Ball bar testing and regular calibration of your machine will help in holding tight tolerances.

5. Calibration for Holding Tight Tolerances

Calibrate your equipment – When you’ve done all of the above but you need it to be just *that* much tighter, consider calling in the manufacturer. After a machine has been built, shipped, dropped off a truck, moved around, leveled, and used for thousands of hours, things will shift and settle. It’s unavoidable. Luckily, there are several pieces of equipment, be it granite squares or the Renishaw Ballbar, that can help pull the reins in on your loosened-up machine to help in holding tight tolerances. We like to perform a ballbar test and make adjustments as part of a yearly maintenance, that way you can keep a tight leash on your machine accuracy. Also, performing these annual services ensures that bearings are tight and lubricated, belts are properly tensioned, and drive motors are healthy – all important factors in having an accurate machine.

Linear scales assist in accuracy and holding tight tolerances.
Linear scales add to a machine’s precision and consistency in holding tight tolerances.

6. Linear Scales for Holding Tight Tolerances

If all else fails, scales! – If you have done everything on this list, and you still struggle, it may be time to consider getting a machine with linear scales. Your typical CNC machine will use the drive motor encoder as the primary method for keeping track of its absolute position, but this can be flawed due to imperfections in the ball screw or thermal discrepancies. Linear scales change all that – typically installed at the factory, they consist of two main components – the scale, and the read head. Put simply – the scale is like a highly accurate ruler that the machine can read, constantly comparing and adjusting for deviations. On our M10Pro, this allows for a 25% tighter positioning tolerance, a 20% improvement in repeatability, and a 85% reduction in backlash..

Use the DATRON M10 Pro to assist in holding tight tolerances in CNC milling applications
The DATRON M10 Pro features linear scales for added precision and accuracy.

Hopefully, these tips will help guide you well down the long, winding, bumpy (but still rewarding!) road of high-precision machining and holding tight tolerances.

Learn More about the DATRON M10 Pro:

Download DATRON M10 Pro Brochure


About the Author

Dann Demazure is an Applications Engineer at DATRON Dynamics. His technical expertise makes him an integral part of programming new projects for potential and current customers as well as training them on the use of their equipment.