Consider the height of the umbrellas — that determines how far up or down the fence you'll mount the brackets — and the angle ...

You may build a 6 foot high privacy fence only to find that the next-door neighbors can easily see over when they're lounging ...