Preheat oven to 325℉ and line an 8x8 pan with parchment paper.
In a stand mixer, cream together the coconut oil and sugar until creamy. Add in the powdered sugar and vanilla until well combined.
Mix in the flour and cinnamon. Add your water 1 teaspoon at a time until you have a shortbread dough that sticks together and can be pressed into the pan.
Press half the dough into the prepared pan.
In a small bowl, mix together the filling ingredients until well combined. Spread this evenly over the dough that is pressed into the pan.
On a piece of parchment paper, press the remaining ½ of the dough mixture into a flat square the size of your pan. Flip this large piece of dough over top of the filling and peel the parchment paper away. It's okay if the dough breaks a bit, just use your fingers to mend the dough together so that it completely covers the filling.
Bake for about 34-37 minutes, or until golden brown. Time may vary depending on your oven. Let cool completely.
In a bowl, mix together the icing ingredients, adding water slowly until you have a consistency that you like. Spread this over top of the cooled pop tart squares and allow to harden before slicing and enjoying!