To delete an alert, move your cursor over the alert Title, click the item drop-down arrow, and select Delete Item. The alert will be deleted.
Disable you alert before deleting it. This will ensure that any custom timer jobs and event handlers are removed before the alert is deleted.