Removing JButtons from a JPanel.

Essentially, what I have coded is a puzzle game.

It contains an image , and the image is further divided into 9 pieces which is placed onto 3x3 JButton GridLayout.

I have already coded the division of images with great help from this forum />

The JPanel size for the JButtons relies on the image. But I do not want the image to be shown at the start. So, when the user first opens the game, the JButton should be empty.

Since the JPanel size relies on the image, the JPanel would be small in size as there will be no image at the start. I used setPreferredSize for the initial empty buttons to solve this problem.

But now, when I want to add the imaged buttons to replace the empty buttons when the user clicks "Start Game" , it doesn't work.

I believe this is due to the setPreferredSize I set for the initial empty buttons.

I tried to remove the initial buttons via
1. puzpiece.remove(button);
2. puzpiece.button.setVisible(false)
3. puzpiece.setPreferredSize(null); ( in the hope that this will overwrite the initial setPreferredSize . )

public class GameFrame extends JFrame implements ActionListener {
    private JButton button1;
    private JButton[] button = new JButton[9];
    private Insets buttonMargin;
    private boolean testImageMethod;
    private JPanel puzpiece;

    public GameFrame(){
        //.. coding ..

        // create new buttons - button1
        button1  = new JButton("Start Game");
        // add action event to "Start" button
        button1.addActionListener(this);

        // initialize instance variables
        buttonMargin = new Insets( 0, 0, 0, 0 );

        // creates a new panel for the splitted puzzle pieces
        puzpiece = new JPanel();
        puzpiece.setLayout(new GridLayout(3,3));

        // check if testImageMethod boolean ( in setupImage() ) is true, 
        //if it isn't, adds 9 buttons w/o images.
         for(int a=0; a<9; a++){
             if(testImageMethod){
             }
             else{
                 // adds 9 buttons without images 
               button[a] = new JButton();  
               puzpiece.add(button[a]);
               puzpiece.setPreferredSize(new Dimension(500,200));
             }

             
         }
         // adds puzpiece panel into the frame 
          this.add(puzpiece,BorderLayout.WEST);     
        //.. coding ..

    }

    public void actionPerformed(ActionEvent e){
        if (e.getSource() == button1){
            // puzpiece.button.setVisible(false);
            //puzpiece.remove(button);
            setImage();

            for(int a=0; a<9; a++){
            puzpiece.add(button[a]);
            }
        }
        else{
            System.out.println("bbb");
        }
    } 

    // method setImage() divides the image into subimages
    public void setImage(){
        //.. coding ..
        setupImage( count++, sc );
    }

    // method setupImage() adds the subimages to the buttons
    private void setupImage( int a, Image wi )
    {   testImageMethod = true;
        button[a] = new JButton( new ImageIcon( wi ) );
        button[a].setMargin( buttonMargin );

    } // end method setupImage()
}