Help me out with some weird insects found on my window & in house

Hello All,

Today I have found many insects like this climbing all over my window from bottom to top & even find few walking around inside my house on floor.
Also there is no garbage container nearby which could possibly cause this. What I have seen is there is some small home at bottom of wondow & I suspect these all are getting out of that hole & climbing to the top of my window.

Any suggestions to get rid of these? Many thanks