Neutron stars change their structure with accumulation of dark matter. We study how their mass is influenced from the environment. Close to the sun, the dark matter accretion from the neutron star does not have any effect on it. Moving towards the galactic center, the density increase in dark matter results in increased accretion. At distances of some fraction of a parsec, the neutron star acquire enough dark matter to have its structure changed. We show that the neutron star mass decreases going towards the galactic centre, and that dark matter accumulation beyond a critical value collapses the neutron star into a black hole. Calculations cover cases varying the dark matter particle mass, self-interaction strength, and ratio between the pressure of dark matter and ordinary matter. This allow us to constrain the interaction cross section, σdm, between nucleons and dark matter particles, as well as the dark matter self-interaction cross section.