let the damn pan soak in hot water for a sec and then you can just friggin wipe them out.
you should probably just start from scratch, scrub the pan down to the iron, then season it properly a couple times. Then you wont have this issue.
And going forward you don't have to get the thing spotless after each use. That's the whole point actually, it's supposed to collect burned grease coating over time. As long as your food gets hot enough to actually cook it you're not dealing with any food safety issues.