Highlight the ones you want to change, right-click, select the first option which is "Get Info", it will ask you if you want to change info for multiple items and once and you click "Yes", then go to the little box that says "Artwork", right-click, and paste.
To change cover art on individual tracks (like if you want to give the specific cover art to singles off an album), you can either right-click-paste to the blank box in the lower left of the main screen which says "Drag Album Artwork Here" when the track is playing--or you can right-click the track, hit "Get Info", go to the "Artwork" tab, and then right-click-paste or click "Add" to browse to a file.
It sounds more complicated than it actually is.