I wasn't sure if I need to create a project first, or just a proposal.
You must first create the Project. Then there are 2 ways to create a proposal:
1) directly from the Proposals List page (video here)
2) go to the Project Work page, create or add from Items there, and use 'Project Options' (video here)
So you can either 1) create a new proposal and add new items directly to it or 2) add existing Items to a new or existing proposal.