Lord Vishnu, by taking the incarnation of a boar (Baraha) protected the earth from being submerged into Patal (underworld) with his long tusk. Then the Lord sat with his wife Laxmi at the bank of Koshi River and the lap of Himalayas and hills. So, the place bore its name after that event. There is a big and beautiful image of the Baraha incarnation of Lord Vishnu.
It is believed that all wishes are fulfilled during life and one gets the eternal grace from Lord Vishnu after death if one takes holy dips at the confluence of Koshi and Koka rivers and worships god at the temple. Shradda performed at this place has greater value than that performed at Gaya.