Short prokaryotic Ago accounts for most prokaryotic Argonaute proteins (pAgos) and is involved in defending bacteria against invading nucleic acids. Short pAgo associated with TIR-APAZ (SPARTA) has been shown to oligomerize and deplete NAD upon guide-mediated target DNA recognition. However, the molecular basis of SPARTA inhibition and activation remains unknown. In this study, we determined the cryogenic electron microscopy structures of Crenotalea thermophila SPARTA in its inhibited, transient and activated states. The SPARTA monomer is auto-inhibited by its acidic tail, which occupies the guide-target binding channel. Guide-mediated target binding expels this acidic tail and triggers substantial conformational changes to expose the Ago-Ago dimerization interface. As a result, SPARTA assembles into an active tetramer, where the four TIR domains are rearranged and packed to form NADase active sites. Together with biochemical evidence, our results provide a panoramic vision explaining SPARTA auto-inhibition and activation and expand understanding of pAgo-mediated bacterial defense systems.