You can delete the element that is currently selected by clicking the 
Delete button on the Home tab of the ribbon or by pressing the Delete key with the exception that the Delete key won't delete the element in the case the Element Properties window is open.
