Sounds like the mill table is not perpendicular to the spindle on at least one plane.

Mount a test indicator in the spindle (on an extension) and sweep the table surface.

If that measures OK...