If you remove specific image size, that won’t be available for future images. It will switch to full sized image – what is the issue in this regard?
When you change image sizes and want that reflect on live site – you need to regenerate all thumbnails. You can use this plugin for it – https://wordpress.org/plugins/regenerate-thumbnails/