No, you do not have to keep your computer running. You can access your vps any time with Remote Desktop Connection &/or putty
The Company will hold your VPS until payment is made, eventually if it's unpaid for a while they usually clean it and re-sell it.
Download Eclipse.
I've been rolling this with company for a while and have no complaints. VPS Hosting, Windows VPS and Linux VPS