is it possible to change my music and my videos icons?

Do you mean on iTunes? I haven't figured it out yet, but if you have a blank icon on itunes just drag a picture there and that should work. Sorry if this wasn't what you were asking.

If I'm not mistaken, I think you can only change the icons for programs.

I'm not on Windows right now, but I think you can right-click and select Properties, and there should be a Customization tab which you can use to change the icon.