Disillusioned,
A routine, post-op CT should show a pretty accurate representation of bone density and healing. Obviously, there's a standard timeline where bones are supposed to be fully healed, but everyone is just so damn different. Even if the CT looks perfect, and your surgeon gives you an OK, I would err on the side of caution, and most importantly listen to your own body. If your CT is great, but it still hurts when you press on your face to wash it in the morning (at night, both, whatever), then I would wait!
Have you ever broken a bone before? This is probably a poor example, but I broke a bone many years ago that healed "alarmingly quickly" (WTF?) according to my doctor. He said I was "100% good to go" at the three week mark (three weeks!). Yes, it looked perfect on my scans, but it also still hurt (maybe the word hurt is a little dramatic, but it definitely didn't feel right). Even though it was technically completely healed, I wouldn't have risked being struck where the fracture took place just yet.
Listen to your surgeon, look at the scans yourself, still be cautious. That's just my two cents.