A small but vocal group is forming new funds and taking new approaches to counter the swell of money into venture capital in recent years.