The chicken can be found in Whiterun. She is usually lurking around the wooden barrel near The Warmaiden's. She'll follow you almost anywhere, even into battle, and will even aggressively rush towards your enemies, but she doesn't do them any damage. Enemies will attack her, but she's essential so she won't die. She uses an animal slot so you can still have a human follower.

Need an egg? Punch her! I know some people are very sensitive, so I want you to know that I don't condone animal cruelty. I thought it would be cool if she could lay eggs, but she doesn't trigger any animation events that I could link to. However, taking a hit does trigger an event. So punching it is. If you are not comfortable with punching fake game chickens, then just don't punch her.

Updates:
V3 - Added inital "Follow me" dialog. Punching chicken gets an egg.
V2 - MasterZipZero has kindly added scripts to dismiss the chicken and ask it to wait! It doesn't work correctly all the time for me, but she also linked me to the tutorial video and I've double-checked and everything looks right, so I would be interested to know if it's just me or if everyone has issues.

Manual Installation
1. Copy ChickenFollower.esp and ChickenFollower.bsa into your Data folder.
2. Launch game. Go into Data Files and make sure ChickenFollower.esp is checked.

To get dialog working, you will need to launch your save game, then make a fresh save and reload the save you just made.

If you should ever find yourself unable to select any dialog options, invoking a load screen by changing locations should fix it.

If telling the chicken to "Go back home" doesn't dismiss her, you can try, hitting tilde (~) key to get into console, and type:

set playeranimalcount to 0