I think there will be two competing forces affecting the price of PM's.
On one hand, the risk off trade where money flows into safe haven assets like fixed income and PM's. This will be more of a "fear" component.
On other hand, Brexit will have trickle down effects. For example, there are currently 800k polish workers in the UK that will now need visas. These workers repatriate their earnings back to families in Poland. Remove that flow of money and you have weakening Zloty, Euro, Czech Koruna. So what strengthens? The dollar and yen. And this will have potential downward pressure on silver.