(1) Background: Prostate cancer is the most common cancer in men and can be treated with radical prostatectomy (RPE) or radiotherapy in the primary setting. Stereotactic radiotherapy (SBRT) has proven to be effective and well tolerated in this setting. However, if SBRT is an equally promising treatment option if applied in the adjuvant or salvage setting after RPE remains unknown. (2) Methods: We searched the PubMed and Embase databases with the following full-text queries in August 2021 for any combination of the terms "SBRT", "prostate", "adjuvant", "postoperative", "salvage", "stereotactic radiotherapy", "prostate bed". There were no limitations regarding publication date or language. We adhered to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) recommendations. (3) Results: We identified 11 individual studies that were included in this systematic review. Three publications included patients without prior radiotherapy and the remaining eight patients with prior radiotherapy. In all but two publications the radiation target was the macroscopic recurrence. SBRT was overall well tolerated with acceptable rates of acute and late gastrointestinal or genitourinary toxicity. Quality of life was published for two phase I trials with good results. There was a very heterogeneous reporting on biochemical control after SBRT. (4) Conclusions: At this point, ultra-hypofractionated RT using SBRT to the prostate bed remains experimental and its use should be restricted to clinical trials. Given the biological rationale for extreme hypofractionation in patients with prostate cancer and the acceptable toxicity rates that have been reported, further exploration of this field is warranted.