The secondary definitely won't be as good when the Pats aren't getting pressure. But of course that's true for every team.
Just FWIW, here are the main DBs' stats this year when thrown against:
Jackson: 75 targets, 50.7%, 6.4 yds/target, 1 td, 7 int, 36.4 rating
Mills: 48 targets, 58.3%, 7.0 yds/target, 5 td, 0 int, 114.8 rating
Bryant: 24 targets, 70.8%, 8.1 yds/target, 0 td, 0 int, 95.0 rating
Williams: 18 targets, 55.6%, 7.2 yds/target, 1 td, 0 int, 96.8 rating
Dugger: 44 targets, 65.9%, 5.8 yds/target, 3 td, 3 int, 75.6 rating
Phillips: 38 targets, 60.5%, 5.9 yds/target, 3 td, 4 int, 63.8 rating
McCourty: 19 targets, 36.8%, 7.5 yds/target, 0 td, 2 int, 24.3 rating
Throwing against NE's safeties is a very bad idea. Throwing at Jackson is a very bad idea. The other corners, however, can be picked on.