Place Oreos in a Ziploc bag and use a rolling pin or heavy object to crush Oreos into small pieces. Set aside.
Combine sweetened condensed milk and dry vanilla cake mix. Stir until cake mix is incorporated into the condensed milk.
Add Cool Whip and mix until well combined.
Sprinkle a little bit of Oreo and sprinkles into the bottom of each cup.
Pour remaining Oreos and sprinkles into ice cream batter. Mix until smooth.
Divide ice cream evenly between cups. Put a straw down the middle of each cup (you may want to cut the straws in half).
Freeze for 3-4 hours to harden. Remove paper lining before eating.