After she passed away, I took it upon myself to try and find the secret to her heart, Grandma Exie Mae Garner's Sweet Potato Pudding. Holiday Yam Pudding I loved trips with Grandma to the country store. Looking to recreate a recipe near-and-dear to her rice. My grandmother left her mark on me in many ways. Add brown sugar, butter, flour, baking soda, salt, and spices.