If you trim or cut that tree down, the downside could be huge. You could be arrested and put in jail for felony destruction of property. Or you could be sued for tens of thousands of dollars. I have seen both scenarios in real life. You will have no defense when they find you, as you are the only one with the motive to do so and the police will figure it out immediately when they see the billboard.
I once got a call from the police that they were going to come arrest me for cutting down a tree next to one of my billboards. Forotunately, I new my signs really well, and I explained to the police that the tree in question would have had no impact on my billboard, but would have a serious blockage issue with a competitor's sign. It turned out I was right, and the other sign owner confessed when the police called. They ended up plea bargaining and paying about $10,000 in fines.
An interesting note is that this same sign owner had deliberately called the city a couple days earlier and used my name as wanting to get approval to cut the tree down -- that's how the police found me. He had tried to frame me.