From my experience, if you have between a certain ping number you are out of range and will constantly disconnect because the signal isn't strong enough to be consistent. For me this has been the case between different countries where I'm in one and the server is in the other so I can't join. But this was for a server by a small group of people usually.
LAN however I've not had much issues with but the LAN was with a family member in the same building usually compared to just a direct connect or otherwise with a friend/online people that are OK with server joiners or that one time I joined a mod author I was familiar with that I've helped in the past. Whilst a server join in the same country has been fine. My difficulties have been with joining I think servers in the US (not sure which side of the country/state for reference), whilst I'm in Australia and my ping can sometimes reach but not too much to be in the world much longer than a few seconds before disconnecting.
I might try joining to see what the situation could be for testing since who knows what the distance is between you both but otherwise I'm not sure. There is a particular range that is clearly not worth bothering with for connecting between users I just don't know right off the top of my head and I'm no expert at networking or servers here myself.