Strap in. Time for some science.
Vaccines work by inducing an immune response in the body to generate antibodies against a particular microbe before an individual is exposed to said microbe. This is done by injecting either a weakened strain of the microbe or by injecting components of the microbe capable of stimulating antibody response. Normally this in itself shouldn't cause any harm. The weakened microbes are incapable of causing disease (theoretically) and the microbial components are merely that, components which alone can't cause disease either. What people are usually worried about are the other components in vaccines, known as adjuvants. Adjuvants are additives that assist the immune system in producing an immunological response to vaccines. Vaccines with adjuvants do have some additional risks compared to those without, such as immune-mediated reactions like inflammation. There's a whole lot more to vaccine safety, so if you want to and if you have the time, you could read this paper which was published 5 years ago.
As for the matter at hand, I think the concern is that a potential vaccine could be rushed, and thus its full scope of action on the human body may not be thoroughly investigated. I can understand why some people would be wary of a COVID-19 vaccine, but considering the extreme contagiousness of the disease and the possibility of death, not to mention the fact that we don't have a treatment for it yet, I think a vaccine might be the only option at this point.
However, not everyone would need to be vaccinated. Herd immunity can be achieved if a sufficient proportion of the population is vaccinated. So even if some people choose not to vaccinate, as long as a sufficient number of people do, society as a whole could theoretically, be able to eradicate the virus. But this poses another conundrum: Who should be vaccinated and who shouldn't? Who should decide?
And for the record, I'm neither pro- nor anti-vaccination. It has to be assessed on a case by case basis. Some vaccines are necessary, some may not be. You can't and shouldn't be all in either way. My undergraduate thesis was entitled "Changes in antibody concentration after vaccination and its possible impact on human health". Here's the abstract from my thesis.
The human body functions through complicated yet balanced interactions between several sophisticated biological systems, such as the digestive system, respiratory system and immune system. Components of the immune system respond to any unwanted external influences, and an incredible amount of molecular elements such as hormones, cytokines and proteins inside immune cells work in synergy to maintain homeostasis and the effective response. Vaccination has been shown to be effective in empowering the immune system to prevent many lethal infectious diseases as well as eradicate some. An activated immune response either by unwanted external influences such as infection, or by vaccination will cause an increase in the rate of production of antibody, cytokines and the related immune cell population. It is not unlikely that these changes will influence the overall homeostasis of the immune system, more specifically, as this study intends to highlight, the dynamics of antibody production. In this study, data was collected from 30 published research articles and the data was used to generate ranges of change in antibody concentrations post-vaccination The ranges generated are as follows: Children - % change in IgG - <0.01-2.27; % change in total antibody - <0.01-2.40; Adults - % change in IgG - <0.01-4.60, % change in total antibody - 0.02-1.65. As it might be possible that low-level immune responses, and not high levels of circulating antibodies are responsible for long-term immunity, vaccination which hyperinduce the immune system may not be necessary, considering that other methods to maintain health are available.
*IgG is a sub-type of antibodies, responsible for long-term immunity.