The first detailed chemical abundance analysis of the M dwarf (M4.0) exoplanet-hosting star Ross 128 is presented here, based upon near-infrared (1.5-1.7 um) high-resolution (R~22,500) spectra from the SDSS-APOGEE survey. We determined precise atmospheric parameters Teff=3231+-100K, log g=4.96+-0.11 dex and chemical abundances of eight elements (C, O, Mg, Al, K, Ca, Ti, and Fe), finding Ross 128 to have near solar metallicity ([Fe/H] = +0.03+-0.09 dex). The derived results were obtained via spectral synthesis (1-D LTE) adopting both MARCS and PHOENIX model atmospheres; stellar parameters and chemical abundances derived from the different adopted models do not show significant offsets. Mass-radius modeling of Ross 128b indicate that it lies below the pure rock composition curve, suggesting that it contains a mixture of rock and iron, with the relative amounts of each set by the ratio of Fe/Mg. If Ross 128b formed with a sub-solar Si/Mg ratio, and assuming the planet's composition matches that of the host-star, it likely has a larger core size relative to the Earth. The derived planetary parameters - insolation flux (S_earth=1.79+-0.26) and equilibrium temperature (Teq=294+-10K) - support previous findings that Ross 128b is a temperate exoplanet in the inner edge of the habitable zone.