You can use DMM to check the speaker's ohm/resistance. Set it to low ohm measurement and it should work fine. You will notice that it is almost impossible to get 4ohms exactly. Sometimes it's pretty close, other times, it's off by 2 or 3 ohms.
Yup. Make sure to set the meter to DCR. Usually the DCR will be a little lower than the rated impedance, example most 4 ohm speakers are actually closer to 3, most 8 ohm drivers are 5-6, and so forth. Not every case, but very common.