Taylor Kanrinin was influenced by a novel that he was reading, and started thinking there should be a god to destroy evil. He tried lots of experiments, and finally created the god Eibee in January 2005. The same year in June, Yagi decided to look for Eibee. They changed their name to Abiroji.
(...Check their official website for more on this crazy story, including some cool animations.)