You can delete the element that is currently selected by clicking the Delete button on the Home tab of the ribbon or by pressing the Delete key with the exception that the Delete key won't delete the element in the case the Element Properties window is open.