You don’t have to accuse yourself… You don’t have to think that you have a poor memory when compared to other speakers. It is easy to manage this problem by scripting your speech effectively. You can write your speech script in a way to easily memorize. How do you do that?
Here is a simple technique. A 5 – 7 minutes speech should focus on just one important message. You can have 2 or 3 key points to convey your message. Each of those key points can have a small story or an example that can stick in your mind (as well as the audience’s). The more stories and examples you have in your speech, it is easier for you to memorize it. Let us take academics for example. We tend to remember the stories that we read during our childhood days, even after several years… but we easily forget the physics or chemistry that we read during the same time.
It is easier to remember stories than just "text". You can still remember some of the stories told during your childhood days because of the way in which it was delivered to you. To the extent possible, include animations in your delivery (hand gestures, stage movements, vocal variety etc.,).
Try scripting your next speech with stories and examples. See the magic it does to you and to your audience.