Background: Quarantine and physical distancing represent the two most important non-pharmaceutical actions to contain the COVID-19 pandemic. Comparatively little is known about possible adverse consequences of these behavioural measures in Germany. This study aimed at investigating potential early adverse effects associated with quarantine and physical distancing at the beginning of the countrywide lockdown in Germany in March 2020. Method: Using a cross-sectional online survey (N = 4,268), adverse consequences attributed to physical distancing, symptoms of psychopathology, and sociodemographic variables were explored in the total sample as well as in high-risk groups (i.e., people with a physical or mental condition). Results: The most frequently reported adverse effects were impairment of spare time activities, job-related impairment, and adverse emotional effects (e.g., worries, sadness). Participants with a mental disorder reported the highest levels of adverse consequences (across all domains) compared to participants with a physical disease or participants without any mental or physical condition. No significant association between the duration of the behavioural protective measures and the severity of adverse mental health effects was observed. Conclusion: Results showed that non-pharmaceutical actions were associated with adverse effects, particularly in people with mental disorders. The findings are of relevance for tailoring support to special at-risk groups in times of behavioural preventive strategies.