England, who have many players from the Great Britain side that won the tournament last year, need to defeat Australia on Saturday to play in the 2016 finale on Sunday.
"We have to be more consistent because the two goals we conceded were individual errors. More concerning it more than one player," said Jon Bleby.
"I expect a tougher match against Australia. We are preparing for Junior World Cup and we need to find consistency."
Though England scored four times, the win did not come easy against a Kiwi side that have struggled in the tournament thus far. New Zealand even appeared to stage a comeback after going three-goals down.
England were two up by the half-time hooter with field goals by Jack Clee and Peter Scott in the 11th and 20th minute respectively. Edward Way made it 3-0 with a penalty corner conversion in the 40th minute.
The Kiwis struck back through Aidan Sarikaya and Dylan Thomas's fields goals in the 52nd and 57th minute to reduce the deficit to one goal, but England's Christopher Proctor restored the two-goal advantage shortly after.
"There is a lot of improvement needed. We must work much harder in our final group game and the Junior World Cup," said New Zealand coach Bryce Collins.