Would help if you gave some context, as this can be caused by quite a few reasons.
How many repositories do you have? How many untracked indexes do they have combined? If you are moving around files quite often, renaming types (or possibly deleting), yet you aren't staging these changes, untracked files can easily pile up. Do this across a few repos and you got yourself a memory issue.
If this isn't the case, you may want to include more details about the enviornment you are working in (How many repositories? How many unstaged changes in each? Etc...)