No, that works fine. With those limitations, you can still build a 8' x 32' -- which is a decent size -- 4' off the ground. You would have two concerns: 1) is that high enough to be seen over any obstructions on that location and 2) will it be vandalized. On concern #2, if you are out in the country, you are probably fine. If you are in a more urban area, you may end up with frequent graffitti.
One item of clarification from the county: is that 12' measured from the ground or the road in which it is viewed? Huge difference. If it is from the viewing road, then you could build a much higher sign.