St. Joseph was founded by German immigrants in 1856 and began as a mission church of Old St. Mary's. The original structure was a wooden frame church built at the corner of Gratiot and Orleans, but by 1869 it became clear that it was inadequate for the congregation.

The present church, designed in the style of a southern German "Hallenkirche" (hall church) by German-born architect Francis Himpler, was begun in 1870. The idea was to celebrate not only God, but German-American culture, creating a welcoming and familiar place for worship. The church still offers Mass in German. The church is a magnificent and largely unaltered example of Victorian Gothic Revival at its best. It is built of Trenton limestone, the last of the twenty-five year cycle of native limestone churches in Gothic revival style in Detroit.

The ornate details, such as the high altar, are noteworthy in that they are entirely carved from wood. Some wood carvings were imported from Germany, but a parishioner, Anthony Osebold, also made some of them. The rebuilt tracker organ displaying the original detail work on the pipes and its bells also makes St. Joseph a fine example of Victorian Gothic revival. The building is a protected structure under the City of Detroit's Historic District Ordinance and is also listed on the Michigan Register of Historic Sites.

St. Joseph’s was dedicated Nov. 16, 1873, though its spire was not completed until 1892. Nevertheless, its bell tower made the church the tallest building in Detroit when it opened.

Other than a loving restoration for its centennial, this stunning church has not changed much since.

But what truly makes St. Joe’s remarkable is its stained glass, which is a combination of imported and local artistry. Much of the glass in St. Joseph was made by the Detroit firm of Friedericks and Staffin, the forerunner of the Detroit Stained Glass Works which operated in Detroit until 1970. But it also includes the earliest known work in North America of the renowned firm Mayer of Munich. The stained glass is so important, in fact, that the church was given the national level of significance by the National Register of Historic Places in 1972. It is the only church in Detroit with that distinction.

As Detroit’s Catholic population has decreased, St. Joe’s was clustered with those of nearby Sweetest Heart of Mary and St. Josaphat into the Mother of Divine Mercy Parish.

In spring 2016, a wind storm severely damaged the historic steeple, forcing the church building to close for several months until emergency repairs could be conducted and the steeple's slate cladding could be removed for safety.

That fall, Archbishop Allen Vigneron announced that St. Joseph's would separate from Mother of Divine Mercy Parish to become St. Joseph Oratory. Under this arrangement, the church would be under the care of the Institute of Christ the King Sovereign Priest and be dedicated to celebrating the traditional Latin Mass.

In October 2017, a $2.5 million historic renewal campaign was begun, helping to ensure it would continue to serve the east side. The damaged steeple's wood underlayment was replaced, and real slate and copper were installed by Detroit Cornice and Slate. Work was completed in September 2019.

Perhaps the damage was a blessing in disguise, as the campaign not only raised money but interest in the church. Recently facing closure, now some 600 people regularly attend Sunday services, a far cry from turnout just a few years earlier.

On March 19, 2020, the church was granted the title of "Archdiocesan Shrine," in recognition of the parish’s service.