When trying to hit a size with a drill, you have to step drill it.  Spot with a center drill, and then drill through with at least 2 sizes smaller.  Then you drill with the size you want.  Sort of a reaming operation with a drill.  That will still leave you slightly oversize - a 1/8" drill will make about a .127 to .129 hole - but it should be straighter and smoother.
I use the rule of thumb that a drill size leaves a running fit hole.  If you want a press fit, you go one size smaller.  I rarely use reamers except on bores where I need a smooth hole.
			

 ) and that marginally improved the initial cut.
 ) and that marginally improved the initial cut.
 In general, it was not my day.
   In general, it was not my day.   
  
 