Although I’m anti-folders and I like to use columns as a way to organize things – there is a limit to how many things you can put in a folder before SharePoint performance starts to degrade. I have always preferred 1000 although some say you can go as high as 2000 items per folder before you have issues. There’s a good article on scaling the performance of SharePoint here.
You can have up to 5 million files comfortably in a document library – but you must use folders. You should set your views to show less than 2000 items. This can be difficult, but remember that your views don’t have to show folders. A flat view can allow you to use folders for organization but still display without folders when needed.
Also remember that you can turn on indexing to improve performance as well.