It is not uncommon for someone to "make the rounds" to people they loved or cared about after death has occurred.
Many times, it is an acknowledgment to you that you are important to them; even if there was not a "closeness". It is also a way of showing that they are "okay" now. And the appearance does not have to happen in a familiar place.
While some can be scared when these visits take place, I consider it to be something that can be used in the ease of the ones still living that the person who passed is not suffering anymore and that they loved you enough to say "see you later".
I was a child when my father passed. He had never been in the house we lived in, but the day he passed, his physical body was in a Veterans Hospital/Home in Florida and I was in New Jersey. He appeared in the way I knew him best and was most used to seeing him as before he got sick - in full uniform (US Air Force). There was a peace on his face. He was no longer trapped inside a prison of a body that required 24/7 care keep him alive.
What a blessing it is to you that your grandfather loved you that much to come to you even if that "closeness" wasn't there. A thing to remember is, especially with older generations who pass is that showing affection and/or expressing affection were not always strong suits in life, but it does not dismiss the internal love that they felt for someone.
Thank you for sharing your experience.