It is just the amount of data you can store. The Difference in performance is in the 3G or 3GS version. I would recommend to spend the extra $100 to get the 3GS. (Unless, that puts your budget over and you won't buy an iPhone at all) First and foremost make sure you buy an iPhone.
To your question I have a 16Gb Phone and it is less than 1/2 full. Don't spend the extra money on the 32Gb size. Lastly. IF you can ONLY afford a refurbished one. OK...but if you can swing a new one. Get a new one. I have had the pleasure of opening the brand new box of an iPhone about 5 or 6 times (my few and my families) It is a great user experience and feeling to be able to put the first scratch on it yourself. :-) Good Luck!