There should be a wiring harness you can buy that will allow you to bypass the amp. I would highly suggest it, the sound will be better and it's very possible the volume will be too loud if you don't bypass it as the gain on the factory amp cannot be adjusted (and will already be set to a high sensitivity since factory head units put out relatively low power).
Best bet would be to Google up a couple of Camry forums and look and ask around. I had to bypass the amp in my '04 Grand Am and it was WAY more of a pain than I had expected. I used the Grand AM Owners Club as a major resource and it was very helpful. Not to scare you though, sometimes it's as easy as the amp bypass harness that was mentioned earlier, that just wasn't the case for me.