In the mid-19th century, Panagiotis Pamouktsoglou settled in Bouloustra, which is how Abdera used to be called, from Constantinople, because he received land for his military services in the Ottoman army. He built his mansion based on traditional architecture and in the type of a castle-house. In the years that followed, he started working on the cultivation and trade of tobacco.

Due to his prestige, as one of the wealthiest people in the village, Pamouktsoglou gathered the money of his fellow villagers, in order to buy land in their name from the Ottoman state. The transaction was accomplished with aid of his son, Kostas, who at that time was studying in Constantinople.

Pamouktsoglou took advantage of his fellow villagers trust and acquired the land for himself with their money. This deception was the reason for his excommunication by the priest of the local church of Agia Paraskevi. According to tle local tradition, the immoral act of Pamouktsoglou against his fellow villagers, was the cause of his own family's misfortune, as none of his six children could bear any offspring.

Now the mansion houses the Ephorate of Antiquities of Xanthi and is used as a guesthouse and a venue for summer events.