{"about":{"non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","site":"https://codewithpapers.app","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page","syntology":{"site":"https://syntology.ai","developers":"https://syntology.ai/developers","mcp":{"server":"https://syntology.ai/mcp","transport":"streamable-http","server_card":"https://syntology.ai/.well-known/mcp/server-card.json","auth":{"type":"trial token, no account","trial_token":"https://syntology.ai/api/oauth/trial/token","method":"POST","docs":"https://syntology.ai/developers"}},"have":"https://syntology.ai/api/graph/have?x=<method, arXiv id or title> (free, answers coverage only)","paper_base":"https://syntology.ai/paper/","atlas_base":"https://app.syntology.ai/?focus="},"machine_readable":[{"url":"https://codewithpapers.app/llms.txt","what":"the machine catalog: every machine-readable file, counted"},{"url":"https://codewithpapers.app/index/manifest.json","what":"paper-to-code index by arXiv id, with Syntology's counts"},{"url":"https://codewithpapers.app/search/manifest.json","what":"site search index (titles, authors) and its files"},{"url":"https://codewithpapers.app/download","what":"bulk files: Syntology's layer, described there"},{"url":"https://codewithpapers.app/build_manifest.json","what":"the build record: inputs, counts, exclusions, probes"}]},"url":"/paper/knowledge-matters-importance-of-prior","title":"Knowledge Matters: Importance of Prior Information for Optimization","arxiv_id":"1301.4083","date":"2013-01-17","proceeding":null,"authors":["Çağlar Gülçehre","Yoshua Bengio"],"abstract":"We explore the effect of introducing prior information into the intermediate\nlevel of neural networks for a learning task on which all the state-of-the-art\nmachine learning algorithms tested failed to learn. We motivate our work from\nthe hypothesis that humans learn such intermediate concepts from other\nindividuals via a form of supervision or guidance using a curriculum. The\nexperiments we have conducted provide positive evidence in favor of this\nhypothesis. In our experiments, a two-tiered MLP architecture is trained on a\ndataset with 64x64 binary inputs images, each image with three sprites. The\nfinal task is to decide whether all the sprites are the same or one of them is\ndifferent. Sprites are pentomino tetris shapes and they are placed in an image\nwith different locations using scaling and rotation transformations. The first\npart of the two-tiered MLP is pre-trained with intermediate-level targets being\nthe presence of sprites at each location, while the second part takes the\noutput of the first part as input and predicts the final task's target binary\nevent. The two-tiered MLP architecture, with a few tens of thousand examples,\nwas able to learn the task perfectly, whereas all other algorithms (include\nunsupervised pre-training, but also traditional algorithms like SVMs, decision\ntrees and boosting) all perform no better than chance. We hypothesize that the\noptimization difficulty involved when the intermediate pre-training is not\nperformed is due to the {\\em composition} of two highly non-linear tasks. Our\nfindings are also consistent with hypotheses on cultural learning inspired by\nthe observations of optimization problems with deep learning, presumably\nbecause of effective local minima.","url_abs":"http://arxiv.org/abs/1301.4083v6","url_pdf":"http://arxiv.org/pdf/1301.4083v6.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"knowledge-matters-importance-of-prior","repo_url":"https://github.com/caglar/structured_mlp","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"none","reach":{"status":"ok"}}],"tasks":[{"task_slug":"unsupervised-pre-training","task_name":"Unsupervised Pre-training"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/1301.4083","atlas_url":"https://app.syntology.ai/?focus=1301.4083","mcp":null,"developers":"https://syntology.ai/developers","held":true,"reason":"arxiv_id_in_atlas","checked_against":{"atlas_ids":549449,"atlas_ids_sha256":"afd72399dafd9906e9062927f70655952385a3d75488558e1f0484e950ab09b7","read_at":"2026-10-05T01:46:21Z"},"agent_calls":[{"tool":"get_citation_path","arguments":{"paper_1":"1301.4083"},"arguments_in":{"paper_2":"another paper's arXiv id or title"},"call":"get_citation_path(paper_1=\"1301.4083\", paper_2=\"…\")"},{"tool":"get_concepts_for_paper","arguments":{"arxiv_id":"1301.4083"},"call":"get_concepts_for_paper(arxiv_id=\"1301.4083\")"}],"by_repo":[{"repo":"caglar/structured_mlp","kind":"official","samples":0,"ran":0,"instrument":0,"unverified":0}],"by_repo_is":"one row per repository in the archive's code links, Syntology's graph links or the samples; samples 0 means no sample is linked to this paper, a statement about Syntology's coverage, not about the repository: with harvested_for_other_papers present, Syntology harvested the repository for that many other papers, and without it, Syntology harvested nothing from it; placed_by_identical_code counts samples whose link records no repository, placed here because identical code was harvested from this repository for another paper; ran counts executions on a synthesized input and instrument counts failures of Syntology's instrument, not of the code"},"code_links_note":null,"arxiv_metadata":null,"syntology_extracted_results":null,"record_sha256":"767a2b89ca5ee914856f45f89b17911c339fcd67eb92c47871396039cd2af76e","record_changed_at":"2026-09-28","record_changed_at_basis":"first_hashed"}