Brexit to end EU freedom of movement 'by definition'