Depends
Mark Mandell - Gone Round In New Jersey
on what you want for a final surface. Here's why
Oil finishes (poly being one) will continue to "shrink" down onto the surface over time as part of the curing process. What may start out as a 7-9 mil coating can easily wind up as 3 mills of a much more dense film. What you will get as a final surface depends on how you have finished and how long you wait to rub it out.
Case in point is this walnut sideboy

Before I knew/learned better, I attempted to do a filled-pore (piano) finish with 18 coats of poly, each carefully sanded level before the next. I waited about 3 weeks and then started the rub-out. I got a beautiful glass-like finish.
It didn't stay that way however. As the finish continued to cure and shrink, the wood's open pores, filled only with poly, began to show. We decided to leave it as is because the very slight grain and faint joint lines actually enhanced the appearance.
From my experience, if you haven't used an actual pore filler on your Oak, I would either wait a good 3 months before doing your final rub-out or do it now to establish the sheen and then allow the grain to reappear. The risk in waiting is that the bottom of the pores will remain shiny as you "satinate" the surface.