That verse quotes from two passages in the Old Testament. The first quote (the one we are concerned with at the moment) is from Deuteronomy 31:6:
And the Lord will give them over to you, and you shall do to them according to the whole commandment that I have commanded you. Be strong and courageous. Do not fear or be in dread of them, for it is the Lord your God who goes with you. He will not leave you or forsake you.”
Here is a breakdown from Strong's Concordance of what those words mean:
Greek: to send back, relax, loosen; to give up, omit, calm; to leave, not to uphold, to let sink [often translated to mean an act of releasing something]
Hebrew: to sink, relax, sink down, let drop, be disheartened; to sink down; to sink, drop; to sink, relax, abate; to relax, withdraw; to let drop, abandon, relax, refrain, forsake; to let go; to refrain, let alone; to be quiet; to show oneself slack [contexts imply relinquishing control]
Greek: abandon, desert; leave in straits, leave helpless; totally abandoned, utterly forsaken; to leave behind among, to leave surviving [most contexts imply abandonment]
Hebrew: to leave, loose, forsake; to leave; to depart from, leave behind, leave, let alone; to leave, abandon, forsake, neglect, apostatise; to let loose, set free, let go, free [to walk away from something]
To “leave” something means to accidentally relax your grip, to forget to take it with you when you go. To “forsake” means to intentionally abandon something, to turn your back on it and walk away on purpose. God has promised to do NEITHER to His people!