Police estimate that a crowd of 600 to 800 gathered at Harvey Milk Plaza in the Castro.
The protest was organized by Rachel Rognerud of San Francisco. She tells KTVU she's not a member of the transgender community but supports its fight for equal rights.
She posted the event on Facebook and it quickly gained support. The mood at the rally was spirited and one of resistance.
"I'm no less a woman because I'm trans," says Layla Wright of San Francisco as she addressed the crowd.
They say this struggle is about equal rights and the fight for safety and survival in and out of the military.
"Transgender women of color are most at risk for violence, most at risk for being murdered," says Marilyn Murrillo, a transgender woman.
She says she's had a hard time finding a job because of discrimination against the transgender community.
Supporters say gay rights have made significant strides, but that the same can not be said for the rights of the transgender community.
"Transgender people get left behind. From the beginning, transgender people stood up for gays and lesbians . I think it's important that we remember that," says Greg Morris, who identified himself as a gay man.
"For every war that I might march against, I will support every soldier who comes home and make sure they have health care," says Megan Rohrer, a transgender man.
He is a chaplain for the San Francisco Police Department. Rohrer and others say the President's ban leaves many questions unanswered.What will happen to transgender troops serving now? Will transgender veterans lose their health benefits.
"This measure is to save the country money," says Leo Lacayo of San Francisco. He's a Republican and is a supporter of President Trump.
Lacayo says the military and the federal government should not carry the burden of costs from sex change operations.
"The army is not a social experimentation camp. It is to defend the nation from attack," says Lacayo.
“Why don’t we stop giving people in the federal government Viagra. That’s many, many times more expensive than the cumulative cost of transgender health care,” says Theresa Sparks, senior advisor to San Francisco Mayor Ed Lee on Trans Initiatives.
During the rally, hundreds chanted "Out of the bars and into the street."
After listening to numerous speakers, the protestors marched along Market Street to City Hall.
"People need to wake up and realize this administration is targeting different minorities of people," says Garrett Montgomery, who marched after attending the rally
Community leaders say the best way to fight back is to hold protests and urge lawmakers to pass legislation that will protect the transgender community.