Space is a mere 100 km above us. If you just wanted to pop up, (then fall back down), you could probably do it in about a minute. If you wanted to get into orbit, it would take about 8 minutes.
Getting high enough is relatively easy, but to stay in orbit you need to be travelling extremely quickly around the Earth. The International Space Station is travelling at about 8 km/s, and goes around the Earth every 90 minutes.
In order to reach this speed, a rocket once launched will quickly start to lean to one side. As it goes higher in the atmosphere, the rocket will roll over until it is practically horizontal. Because it’s not travelling vertically, it takes a bit longer to reach space.