Outlook plays a chime to alert you to scheduled reminders on email, tasks, or calendar events. Unlike the new mail alert (which you manage in Windows), you manage the reminder alert in Outlook, where you can change the chime to something else or turn it off altogether. Here’s how to get that done.
If you don’t want an auditory alert every time a reminder pops up (which can be especially jarring if you’re working with headphones on), then you can tell Outlook not to play a noise at all when a reminder is due. Go to File > Options > Advanced and scroll down to the “Reminders” section. The setting you’re looking for is “Play Reminder Sound.”