The investigation of the metal-poor tail in the Galactic bulge provides unique information on the early Milky Way assembly and evolution. A chemo-dynamical analysis of 17 very metal-poor stars (VMP, [Fe/H]<-2.0) selected from the Pristine Inner Galaxy Survey was carried out based on Gemini/GRACES spectra. The chemistry suggests that the majority of our stars are very similar to metal-poor stars in the Galactic halo. Orbits calculated from Gaia EDR3 imply these stars are brought into the bulge during the earliest Galactic assembly. Most of our stars have large [Na,Ca/Mg] abundances, and thus show little evidence of enrichment by pair-instability supernovae. Two of our stars (P171457 and P184700) have chemical abundances compatible with second-generation globular cluster stars, suggestive of the presence of ancient and now dissolved globular clusters in the inner Galaxy. One of them (P171457) is extremely metal-poor ([Fe/H]<-3.0) and well below the metallicity floor of globular clusters, which supports the growing evidence for the existence of lower-metallicity globular clusters in the early Universe. A third star (P180956, [Fe/H]~-2) has low [Na,Ca/Mg] and very low [Ba/Fe] for its metallicity, which are consistent with formation in a system polluted by only one or a few low-mass supernovae. Interestingly, its orbit is confined to the Galactic plane, like other very metal-poor stars found in the literature, which have been associated with the earliest building blocks of the Milky Way.