Richards Bay opened the scoring in the 29th minute. Daniel Akpeyi misjudged Khanyisa Mayo's corner-kick allowing Dube to nod home from close range after the ball floated over the outstretched arm of the Chiefs shot-stopper.
Kaizer Chiefs suffered shock Nedbank Cup exit after going down 2-1 to GladAfrica Championship side Richards Bay FC.
Chiefs last won the tournament in 2013, defeating SuperSport United 1-0 after extra-time to clinch the title for the 13th time.
Siyabonga Vilane scored a stoppage-time winner for the Natal Rich Boyz as Chiefs fell to defeat against lower league opposition yet again. Leonardo Castro had levelled matters for the hosts, who had Siphosakhe Ntiya-Ntiya sent off, following Thabani Dube's first-half opener.
In an attempt to breath new life into his team, Gavin Hunt made no fewer than four changes during the halftime break and the Amakhosi gradually improved with Malcolm Jacobs called into action in the 54th minute to keep out Samir Nurkovic's bullet header.
Chiefs have been their own worst enemy for most of the season and were at it again 10 minutes later as Ntiya-Ntiya received his marching orders following a second bookable offence.
But despite being a man down, the Amakhosi were able to level matters through Castro who fired home via the inside of the post while lying flat on his back in the 76th minute.
Castro had a glorious chance to snatch the winner for Amakhosi in the last two minutes after a cross by Zuma towards the far post but he headed his effort over the bar from close range.
Siyabonga Vilane netted the winner for the Richard Bay in stoppage time with a strike into an empty net from a rebound after Beyzel Goldstone went through on goal and hit the post with his strike and booked their spot in the last 16.
Kaizer Chiefs (0) 1 (Castro 76')
Richards Bay FC (1) 2 (Dube 29' Vilane 90+1')
Kaizer Chiefs:Akpeyi; Ntiya-Ntiya, Cardoso, Akumu, Moleko, Matsheke (Blom 46'), Ntshangase (Manyama 46'), Parker (Castro 46'), Mashiane (Zuma 46'), Ngcobo (Frosler 68'), Nurkovic.
Richards Bay FC:Jacobs; Mabua, Dube, Zulu, Maphathe, Bakah, Mtolo, Mthembu (Goldstone 73'), Ogbonna (Maphakisa 52'), Mayo (Mabuya 52'), Ntsundwana (Vilane 74').