Мы не можем просто сидеть сложа руки.
Given the meaning of the expression -- "sit around, twiddling thumbs", or more literally, "sit around, folding arms" or "sit around with folded arms" -- "сложа" seems to function like an adverbial participle, whether it be present or past, as well as perfective or imperfective:
Мы не можем просто сидеть, сложив руки.
Мы не можем просто сидеть, сложивши руки.
Мы не можем просто сидеть, складывая руки.
... even though the third option doesn't sound right to me. I'm wondering if there are other expressions with a peculiar verb form like "сложа"?