Menu
Is free
check in
the main  /  Installation and Setup / WGET commands: examples. Console program for downloading files over the WGET network

WGET commands: Examples. Console program for downloading files over the WGET network

wGET - Guide GNU Wget

SYNTAX

wget [ parameters]… [ URL ]…

DESCRIPTION

GNU WGET is an openly displaced utility for downloading files from the Internet. It supports HTTP, HTTPS, and FTP protocols, download from the http proxy servers.

WGET can follow the links of HTML pages and create local copies of remote Web sites, while it is possible to fully restore the site folder structure ("Recursive Download" - recursive loading). During such work, WGET is looking for a file with access to robots (/robots.txt). It is also possible to convert links to the downloaded HTML files for further viewing of the site offline ("OFF-LINE BROWSING").
Check file headers: WGET can read file headers (this is available by hTTP protocols Both FTP) and compare them with headlines of previously downloaded files, after which you can download new versions of files. Due to this, when using WGET, you can implement mirror storage of sites or file sets to FTP.
Wget is designed for slow or unstable connections: if a problem occurs during the download, the WGET will try to continue the download of the file. If the server from which the file is loaded, supports the file to the file, then the WGET will continue to download the file from the place where the download broke out.

Options

Main settings

-V. --Version. Display the WGET version. -h --Help Display the WGET command line parameters. -b. --BackGround Go to the background mode after starting. If the file for messages is not specified by the parameter -o.then he is written in wget-Log. -E. command. --Execute. command. Perform command. as if it were part .wgetrc.. The command will be executed after the commands in .wgetrc..

Message parameters

-o. logfile. --OutPut-File \u003dlogfile. Write all messages in logfile.. Otherwise, they will be directed to Stderr. -a. logfile. --Append-Output \u003dlogfile. Additional logfile.. Like I. -o., Only logfile. Not replaced, but complemented. If a logfile. There is no, a new file is created. -d. --Debug Display debug messages - Miscellaneous information important to WGET developers. -Q. --Quiet. Turn off the WGET messages. -V. --Verbose Enable detailed messages, with all available data. Default included. -NV. --Non-verbose Weoplize abbreviated messages (to turn off the messages, see. -Q.). Error messages and basic information will be displayed. -I. file --Input-file \u003dfile Read url out file. In this case, you do not need to specify the URL on the command line. If the URL is also specified on the command line and in file, First, the URL will be downloaded from the command line. file It does not have to have an HTML format (but nothing terrible, if so) - the URL simply should be indicated in it. If you specify --Force-HTML.then the file will be read as hTML. In this case, problems with relative references may arise. This can be prevented by adding to it. " "or input on the command line --Base \u003d.uRL. -F. --Force-HTML. When reading the URL from the file, includes reading a file as HTML. To prevent errors in the case of the Local HTML file, add to the file " "Or enter the command line parameter --Base. -B. URL --Base \u003d. URL When reading the URL from the file ( -F.) Determines URL added to the relative file addresses specified by the parameter -I..

Download options

--Bind-Address \u003d Address When connections via TCP / IP transmits "bind ()" in Address on a local machine. IN Address It can be indicated both host name and IP address. Used if your computer has several IP addresses. -t. number --Tries \u003d.number Sets the number of repetitions number. Specify 0 or inf To cancel repeats. -O. file --OutPut-Document \u003dfile Documents will not be recorded in the relevant files, and will be combined together and recorded in the file. file. If a file There is, it will be replaced. If a filespecified as , Documents will be displayed on the standard output (stdout). This parameter automatically sets the number of repetitions in 1. Useful when downloading divided from files from mail servers via the web interface. -nc. --No-clobber If when downloading the site, the connection was cut off, then specify this parameter to continue downloading from the connection site. After starting the WGET without parameters -N., -nc., or -r. Loading the same file in one folder will create a copy of the file with the name file.1 . If there is a file and with the same name, the third copy will be called file.2 etc. Under the parameter -nc. Warnings will be displayed. When you start the WGET with the parameter -r., but without -N. or -nc., The new download site will replace the already loaded files. When specifying the parameter -nc. Loading will continue from the breakdown and the downloaded files will not be loaded again (unless they change). When you start the WGET with the parameter -N., with or without -r.The file will be loaded only if it is newer than an existing one, or if the size does not match the available copy (see comparison by date). -nc. C. is not combined -N..
With the specified parameter -nc. Files with extensions .html or (this is just awful) .htm. From local disks will be loaded, as if from the Internet. -c. --Continue. Renewing the file download. Used if the file download was interrupted. For example: Wget -c ftp://sunsite.doc.ic.ac.uk/ls-lr.z

If the current folder already has a file named lS-LR.Z., then WGET will check whether this file matches the downloadable file (not in size!), and if so, then you will send a request to the server to continue to download the file from the same place where the download broke out last time. Please note that when the connection is broken. WGET Repeats attempts to download yourself and without a parameter -c.And only when he "will give up" and completes its work, then this parameter will be needed to resume the file loading.
Without specifying option -c. The previous example will load the specified file re-with the final name. lS-LR.Z.1., no touch already existing lS-LR.Z..
Starting from version 1.7 when specifying the parameter -c. If the file on the server is equal to or smaller than the local file, the size, the WGET will not download anything and displays the corresponding message.
However, when used -c. Any file on a server having a larger size than a local file will be treated as unattached. It will be downloaded and recorded at the end of the file only "(Length (delete file) - Length (local. File))" Byte. This can be useful, Elsi you need to download new messages from any magazine (LOG).
At the same time if the downloadable file is more because it changedYou will receive a damaged file (i.e. the file may eventually turn out to be completely different from the original). Need to be especially attentive when used -c. together with -r.Since each modified file can be a candidate for "incomplete loading".
You will also receive a damaged file If your HTTP proxy server works stupid and when the connection is broken, it writes to the message file "transfer interrupted". Probably in the following versions of Wget will correct it.
remember, that -c. It works only with FTP and HTTP servers that support the "Range" headers (i.e. file fraud). --Progress \u003d.type Load progress indicator and its type. Possible values: "DOT" and "BAR". According to the default, "BAR" is used. Note option --Progress \u003d Bar. will lead to the drawing of a beautiful indicator of ASCII characters (as "thermometer"). If the standard output is not TTY, then "dot" will be used. --Progress \u003d dot. To switch to the type "DOT". The loading progress will be marked by adding a point or sign in the strip equal to each symbol represents the same amount of data. When using this type, you can specify its style - dot:style.. If the style is "default", then each character will represent 1 kb, 10 characters in the cluster and 50 in the string. The style "binary" has a more »Computer" view - 8Kb in a symbol, 16 characters in a cluster and 48 characters in the string (it turns out a line for 384 Kb). The MEGA style is used to download large files - each symbol represents 64kb, 8 characters in the cluster and 48 characters in the string (3 MB per line is obtained).
You can define the default style using the command "Progress" in .wgetrc.. If you want the type of indicator "BAR" to be used always (and not only when displaying in stdout), then specify --Progress \u003d Bar: Force. -N. --TimeStamping. Enable comparison by date. -S. --Server-Response. Display headers sent HTTP servers and requests sent to FTP servers. --Spider. Installing WGET behavior as a "spider", i.e. wget will not load files, but will only check their presence. So you can check the bookmarks and links of the site. For example:

Wget --spider --Force-Html -i Bookmarks.html

WGET does not contain all the possibilities of "real spiders" for www. -T Seconds. --Timeout \u003d.seconds. Waiting time in seconds. By default, the waiting time is 900 s (15 min). Setting the value to 0 cancels the wait time check. Please, do not reduce the waiting time if you definitely do not know what exactly you are doing. -w. seconds. --Wait \u003d.seconds. Pause in seconds between multiple loads (including repetitions). This reduces server load. To specify a value in minutes, use "M", in the clock - "H", in the days - "D" after the number. When the large value of this parameter is useful if the network is unstable (for example, when the modem communication breaks). --WaitreTry \u003d.seconds. Sets the pause only between reversing loads. WGET will wait 1 second after the first break, 2 seconds after the second cliff of the boot of the same file, etc. - to the maximum, which is indicated in seconds. For example, with the value of this parameter, equal to 10, WGET will wait in general (1 + 2 + ... + 10) \u003d 55 seconds for each file. This value is specified by default in the file wgetrc.. --Random-wait. Some servers, making log file files with file query pauses, can determine the recursive download of files - scan robots such as WGET. This parameter sets the time between requests by varying the pauses with time calculated from 0 to 2 * wait. (seconds), where wait. Specified Parameter -w. To disguise wget. Do not forget that the source code of the WGET is available, and it can even be calculated if you want to calculate this disguise. -Y ON / OFF --Proxy \u003d ON / OFF Support for a proxy server. Enabled by default if the proxy is defined. -Q. quota --Quota \u003d.quota Quota for the size of downloadable files. Is indicated in bytes (by default), in kilobytes KB (if at the end k.) or in MB megabytes (if at the end m.). And the quota exhaustion current file. Loading to the end, that is, the quota does not work when loading one file. For example, if you perform wGET -Q10K. ftp://wuarchive.wustom.edu/ls-lr.gz. then file. lS-LR.GZ. will be fully loaded. Also, all files specified on the command line will be loaded, unlike the file list in one file or as with a recursive load. Wecript 0 or INF will cancel the quota.

Failock boot options

-nd --No-directories. Do not create folder structure with recursive download. With the specified parameter, all files will be loaded into one folder. If the file with this name already exists, it will be saved under the name Name File.n.. -X. --Force-Directories. The opposite of the parameter -nd - create folder structure starting with main page Server. For example, wGET -X http://fly.srk.fer.hr/robots.txt will load the file to the folder fly.Srk.fer.hr.. -Nh. --No-Host-Directories Do not create empty folders At the beginning of the structure. Default / Pub / Xemacs /. If you download it with the parameter -r.then it will be saved under the name ftp.xemacs.org/pub/xemacs/. With parameter -Nh. From the name of the initial folder cuts out ftp.xemacs.org/and she will be called pub / Xemacs.. And parameter --Cut-dirs. Remove number Components. Examples of parameter --Cut-dirs.: Without parameters -\u003e ftp.xemacs.org/pub/xemacs/ -NH -\u003e Pub / Xemacs / -Nh --cut-dirs \u003d 1 -\u003e Xemacs / -Nh --cut-dirs \u003d 2 -\u003e. --Cut-dirs \u003d 1 -\u003e ftp.xemacs.org/xemacs/ ...

If you want to just get rid of the folder structure, then you can replace this parameter on -nd and -P.. Unlike -nd, -nd Works with subdirectories - for example, when -NH --cut-dirs \u003d 1 subdirectory bETA / Wrong, as xemacs / Beta.. -P. prefix. --Directory-Prefix \u003dprefix. Determines initial folderwhere the site folder structure (or just files) will be saved. By default, this parameter is equal . (Current folder).

HTTP Parameters

-E. --Html-extension If the type of download file text / Html. and its address will not end on .? when using this parameter to its name will be added .html. It can be useful when mirrored pages .aspIf you do not want them to interfere in the work of your Apache server. Another case of application of this paramer is to download CGI Script Response Pages. Page with URL http://site.com/article.cgi?25 will be saved as article.cgi? 25.html.. Upon: When updating or another reboot of pages with this parameter, the latter will be loaded in any case, because WGET can not find out whether the local file has the relationship X.html. To loaded with URL X.. To avoid unnecessary reboot, use options. -K. and -K.. In this case, the original versions of files will also be saved as X.orig. --Http-user \u003duser. --Http-passwd \u003dpassword. Username user. and password password. For HTTP server. Depending on the type of response, the WGET will use "Basic" (unsafe) or "Digest" (secure) authorization. You can also specify the username and password and in the URL itself. -C ON / OFF --Cache \u003d ON / OFF Includes or turns off the server caching. In this case, WGET sends the appropriate request ( Pragma: No-Cache). Also used to quickly update files on a proxy server. The default caching is allowed. --Cookies \u003d ON / OFF Includes or turns off the use of cookie. The server sends a cookie to the customer using the "Set-Cookie" header and the client meets the same cookie. Thanks to this, the server can lead the statistics of visitors. By default, cookies are used, but record them on the disk is turned off. --Load-cookies. file Download cookie out file Before the first loading HTTP. file It has text format, as cookies.txt Netscape. This parameter is used for mirroring. To do this, WGET sends the same cookies that sends your browser when connecting to the HTTP server. This is turned on by this parameter - just specify the WGET path to cookies.txt. Different browsers Store cookie in different folders: Netscape 4.x. File is B. ~ / .netscape / cookies.txt. Mozilla and Netscape 6.x. Mozilla stores cookies in cookies.txtlocated somewhere in ~ / .mozilla., In the folder of your profile. The full path usually ends something like ~ / .mozilla / Default / Some Weird-String / Cookies.txt. Internet Explorer. To export a cookie for WGET, select "File", "Import and Export", select "Export Cookie Files" in the Wizard. Verified in Internet Explorer 5; It may not work in earlier versions. Other observers. Parameter --Load-cookies. It will work with cookie in Netscape format that is supported by WGET. If you can not use the parameter --Load-cookies., it still has a way out. If your browser supports write the name and cookie value and manually specify Wget sending to these cookies: Wget --Kookies \u003d OFF --Header "Cookie: I \u003d I. " --Save-cookies file Save cookie out file At the end of the session. Outdated cookies are not saved. --GNore-Length. Some HTTP servers (more precisely, CGI scripts) seize the "Content-Length" headers, which indicate the WGET, which is not yet loaded. And the WGET loads one document several times. With this parameter, the WGET will ignore the "Content-Length" headlines. --Header \u003d.additional-Header. Determines additional-Header.sent to the HTTP server. He must contain : and characters after it. You can define several additional headers through the use --Header. repeatedly. Wget --Header \u003d "Accept-Charset: ISO-8859-2" --Header \u003d "Accept-Language: HR" http://fly.srk.fer.hr/

Specifying an empty string in the title value will clean all user-defined to this headlines. --Proxy-user \u003duser. --Proxy-passwd \u003dpassword. Defines username user. and password Password to authorize the proxy server. The authorization type "Basic" will be used. --Referer \u003d.uRL Adds the title `Referer: uRL'In the HTTP request. Used when downloading pages that are transmitted correctly only if the server knows which page you came from. -s. --Save-Headers. Save headers sent by http servers. -U. agent-String --USER-Agent \u003dagent-String Identify, as agent-String When prompted to HTTP server. The HTTP protocol allows you to determine yourself using the agent header. The default WGET is identified as WGET /version.where version. - This is a version of WGET. Some servers provide the required information only for browsers identifiable as "Mozilla" or Microsoft "Internet Explorer". This parameter allows you to deceive such servers.

Parameters FTP

-Nr. --Dont-Remove-Listing Do not delete temporary files .listing.generated when downloading by FTP. These files contain information about FTP servers folders. Upholstery will help you quickly determine the server folder update (i.e., to determine that your mirror is such). If you do not delete .listing., I remember your safety! For example, with this name, you can create a symbolic link to / etc / passwd or something else. -G ON / OFF --Glob \u003d ON / OFF Includes or turns off the use of special characters ( masks) via ftp protocol. It may be * , ? , [ and ] . For example: Wget ftp://gnjilux.srk.fer.hr/*.msg

By default, the use of mask characters is allowed if the URL contains such characters. You can also take the URL in quotes. It will work only on UNIX FTP servers (and emulating the Unix "Ls" output). - Passive-FTP. Includes FTP passive mode when the connection is initiated by the client. Used with Firewall. --Retr-symlinks. With the recursive download of FTP folders, the files that indicate symbolic links are not loaded. This parameter disables this. Parameter --Retr-symlinks. It is now only for files, not for folders. Please note that this parameter does not work when loading a single file.

Recursive loading parameters

-r. --Recursive. Enable recursive download. -L. depth --Level \u003d.depth Maximum depth of recursive loading depth. By default, its value is 5. --Delete-after. Delete each page (locally) after Its downloads. Used to save new versions of frequently requested pages on a proxy. For example: Wget -R -ND --Delete-After http://whatever.com/~popular/page/

Parameter -r. Includes loading by default, parameter -nd Disables folder creation. Print the specified parameter --Delete-after. Parameter will be ignored --Convert-Links.. -K. --Convert-Links. After the download is completed, convert links to the document to view offline. This applies not only to the visible references to other documents, and links to all external local files. When the link varies with one of two ways:

* Links to files loaded by WGET vary on the appropriate relative links. For example: if the downloaded file /foo/doc.html., then link to also downloaded file /bar/img.gif. will look like ../bar/img.gif.. This method works if there is a visible ratio between the folders of one and another file. * Links to files that are not loaded WGET will be changed to the absolute addresses of these files on the remote server. For example: if the download file /foo/doc.html. Contains link to /bar/img.gif. (or at ../bar/img.gif.), then link in the file doc.html. will change on http: //host/bar/img.gif.. Because of this, it is possible to autonomous viewing of the site and files: if the file is downloaded to which there is a link, the link will indicate it, if not, the link will indicate its address on the Internet (if such exists). When converting relative links are used, it means you can transfer the downloaded site to another folder without changing its structure. Only after the download is complete, the WGET knows which files are loaded. Consequently, when parameter -K. Conversion will occur only upon completion of the download. -K. --Backup-converted Convert links back - remove extension .orig. Changes the behavior of the option -N.. -m. --Mirror Enable parameters for mirror storage sites. This parameter is equal to several parameters: -R -N -L inf -NR. For unpretentious storage of mirror copies of sites, you can use this parameter. -p. --Page-Requisites. Download all files that are needed to display HTML pages. For example: drawings, sound, cascade styles. According to default, such files are not loaded. Parameters -r. and -L.specified together can help but because WGET does not distinguish external and internal documents, then there is no guarantee that everything required. For example, 1.html. Contains a tag " " , with reference to 1.Gif., and tag " "referring to an external document 2.html. Page 2.html Similar, but its drawing - 2.gif.and she refers to 3.html. Let's say it lasts until a certain number. If the command is given: WGET -R -L 2 http: // i /1.html.

that 1.html., 1.Gif., 2.html, 2.gif. and 3.html Download. As you can see 3.html without 3.Gif.because Wget simply considers the number of jumps by which it switched, comes to 2 and stops. And when parameters:

WGET -R -L 2 -P http: // I /1.html.

All files and picture 3.Gif. pages 3.html Download. Similarly

WGET -R -L 1 -P http: // I /1.html.

load to download 1.html., 1.Gif., 2.html and 2.gif.. To upload one specified HTML page with all its items, just do not specify -r. and -L.:

Wget -p http: // i /1.html.

In this case, WGET will behave like when the parameter -r.But the page and its auxiliary files will be downloaded. If you want auxiliary files on other servers (i.e., using absolute links) are loaded, use:

Wget -e -h -k -k -p http: // I / I.

And in conclusion, you need to say that for wget external reference - This is the URL specified in the tags " " , ""And" " , Besides " " .

Recursive Recursive Download / Permissions Parameters

-A. acclist. --Accept acclist. -R. rejlist. --Reject. rejlist. List of file names separated by commas, which should or should not be downloaded. It is allowed to set the file names on the mask. -D. domain-List. --Domains \u003d.domain-List. List of domains domain-List.which are allowed to upload files. Shared commas. This parameter not Includes -H. --Exclude-domains. domain-List. List of domains from which not Allowed to upload files --Follow-FTP. Follow the FTP links from HTML pages. Otherwise, links to FTP files are ignored. --Follow-Tags \u003dlist WGET has a built-in HTML tag table in which it is looking for links to other files. You can specify additional tags in the split list list In this parameter. -G. list --GNore-Tags \u003dlist Back --Follow-Tags.. To skip HTML tags with recursive download, specify them in a split list list. Intrush the parameter -G. was the best for download individual pages With their auxiliary files. You can see how it was, indicating the command wget -GA, Area -h -k -k -r http: // I / I.

But now the best parameter To download one page completely considered --Page-Requisites.. -H --Span-hosts. Allows you to visit any servers to which there is a link. -L. --Relative. Follow only relative links. In this parameter, files from other servers will definitely not be loaded. -I. list --Include-directories \u003dlist The list of folders divided by commas from which it is allowed to upload files. List elements list -X. list --Exclude-directories \u003dlist List of folders divided by commas excluded for download (see folder restriction). List elements list May contain symbols of masks. -np. --No-Parent. Do not rise above the initial address during recursive download.

EXAMPLES OF USING

Examples are divided into three categories according to their complexity.

Just use

* If you need to download the URL, then enter: wget http://fly.srk.fer.hr/ * But what will happen if the connection is slow, and the file is long? There is a possibility of breaking communication before the download is completed. In this case, the WGET will continue to attempt the new connection until the number of attempts (by default 20). You can change this number, for example up to 45: Wget --Tries \u003d 45 http://fly.srk.fer.hr/jpg/flyweb.jpg * Now let's leave Wget to work in the background, and we will record it log log.. Dive --Tries.so we use -t.. Wget -t 45 -o Log http://fly.srk.fer.hr/jpg/flyweb.jpg &

The ampersand symbol at the end indicates the command interpreter to continue the work without waiting for the completion of WGET. To make the program repetitions endlessly - use -t inf. * Use FTP is also very simple. Wget takes all the concerns on authorization.

Wget ftp://gnjilux.srk.fer.hr/welcome.msg * If you specify the folder address, the WGET will load the listing of this folder (i.e. the files and subdirectories contained in it) and converts it to the HTML format. For example: Wget ftp://prep.ai.mit.edu/pub/gnu/ Links index.html

Extended use

* If you have a file with the URL you want to download, then use the parameter -I.: wget -i i

If you specify Instead of file name, the URL will be read from standard input (STDIN). * Create a five-level copy of the GNU website with the structure of the original folders, with one boot attempt, save messages in gnulog.:

Wget -R http://www.gnu.org/ -o gnulog * As in the example above, but with the links in the HTML files to the local, for later autonomous viewing: WGET --Convert-Links -r http: // www.gnu.org/ -o gnulog * Download one HTML page and all the files required to display the latter (eg drawings, cascading styles, etc.). Also convert all links to these files: WGET -P --Convert-Links http://www.server.com/dir/page.html

HTML page will be saved in www.server.com/dir/page.html and drawings, cascading styles and so on will be saved in the folder www.server.com/In addition to the case when the files are downloaded from other servers. * As in the example above, but without a folder www.server.com/. Also everything Files will be saved in subfolders download /.

Wget -p --Convert-Links -Nh -ND -Pdownload http://www.server.com/dir/page.html * Download index.html with www.lycos.com, displaying server headers: wget -s http://www.lycos.com/ * Save headers to a file for further use. Wget -s http://www.lycos.com/ more index.html * Download two higher levels wuarchive.wustom.edu. in / TMP.. Wget -R -L2 -P / TMP ftp://wuarchive.wustom.edu/ * Download GIF files folders on HTTP server. Team wget http://www.server.com/dir/*.gif. It will not work, since the camouflage characters are not supported when loading through the HTTP protocol. Use: Wget -R -L1 --No-Parent -A.gif http://www.server.com/dir/

-R -L1 Includes recursive loading with a maximum depth 1. --No-Parent. Turns off following the links to the parent folder having upper level, -A.gif. Allows you to upload only files with the .gif extension. -A "* .gif" It will also work. * Suppose that during the recursive download you needed to turn off / restart the computer. To not download existing files, use:

Wget -nc -r http://www.gnu.org/ * If you want to specify the username and password for the HTTP server or FTP, use the appropriate URL syntax: Wget FTP: // hniksic: /.emacs * you want to Downloadable documents went to standard output, not in files? Wget -o - http://jagor.srce.hr/ http://www.srce.hr/

If you want to arrange a conveyor and download all sites, links to which are indicated on one page:

Wget -o - TTP: //cool.list.com/ | Wget --Force-HTML -i -

Professional use

* For storing the mirror of the page (or FTP folders), then use --Mirror (-m.) what replaces -R -L info -n. You can add wget to Crontab with a request for update check every Sunday: Crontab 0 0 * * 0 Wget --Mirror http://www.gnu.org/ -o / Home / Me / Weeklog * You also want links converted To local. But after reading this manual, you know that the time comparison will not work. Specify the WGET to leave backup copies of HTML files before conversion. Team: Wget --Mirror --Convert-Links --Backup-converted http://www.gnu.org/ -o / home / me / weeklog * And if local viewing does not work hTML files with an extension other than .html, eg index.cgi., then you need to transfer the command to rename all such files (Content-Type \u003d text / Html.) in name.html.. Wget --Mirror --Convert-Links --Backup-converted --HTML-EXTENSION -O / HOME / ME / WEEKLOG http://www.gnu.org/

With brief analogues of the teams:

Wget -m --k -k -e http://www.gnu.org/ -o / Home / Me / Weeklog

Files

USR / LOCAL / ETC / WGETRC By default, this is the location global File settings. .wgetrc. User settings file.

Found errors

You can send error reports in the GNU WGET to the address< " " > (in English).
Before sending:

1. Make sure the behavior of the program is really wrong. If Wget "crashes", then this is an error. If the behavior of the WGET does not correspond to the documentation, then this is an error. If everything works strange, but you are not sure how it should actually work, then this may also be a mistake. 2. Try to repeat the situation with the issuance of an error for the minimum number of actions. Do not hurry to send .wgetrc., Try to do all the actions that lead to an error with another settings file (or without it). 3. Run WGET with a parameter -d. And send a magazine (or part of it). Far It is easier to search for errors in the presence of such magazines. 4. If WGET issued an error, then try to start it in the debugger, for example, "GDB` Which Wget` Core "and enter" WHERE "to get back tracing.

CM. ALSO

GNU info for wGET..

AUTHORS

TRANSFER

Copyright

Copyright (C) 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004,
2005, 2006, 2007, 2008, 2009, 2011 Free Software Foundation, Inc.
This is free software; Copy conditions are looking for in the source texts. There are no guarantees; The program is not intended for sales.

It would seem in the world where half of the world's population actively uses computers and got used to stylish graphical interfaces and rich opportunities such, there is no place for text applications. Some 20 years ago, the computer interface was a terrible black panel, which did not displays anything except for a set of white characters, and all control was carried out due to text commands. Surprisingly, as far as these programs are ugly, are as functional and reliable.

What is WGET?

Wget (which is also referred to as GNU Wget Linux) is a console, not an interactive application for working with files and web pages. In fact, this is a program to download files from the Internet, characterized by the lack of a graphical interface and a number of features in control, atypical for more modern solutions.

The program was developed by an engineer from Croatia Hrwebi Nicshich, and now the development supports Giuseppe screenshots and the Linux community. The first assembly of the utility (then called GETURL) was presented in January 1996. The program was written in the SI programming language and was published by gNU licenses. Initially, the tool was released on the Linux platform. Over time, the utility has been translated into many languages \u200b\u200band is ported to all popular platforms, including WGET for Windows 7 (and newer), Wget for MacOS and so on.

This program is not interactive, and therefore any process running under its capabilities cannot be controlled, they can be managed by contacting the controls of the operating system itself.

Despite the fact that modern browsers usually have built-in, they are still designed for interactive interaction, therefore work in manual mode It becomes difficult. WGET, on the contrary, allows a number of processes to automate. For example, it is possible to prepare in advance text Document With a list of web pages using which you can download several files and pages at once.

Basic functions of the program

Nevertheless, first of all, this is a program for downloading files, which is the evidence of its main function.

WGET can download any materials from the Internet, whether documents, executive files or entire web pages in HTML format (for various protocols). Files can be downloaded from directories

The client can perform a recursive download, in automatic mode, moving through the links within a specific website (or several websites), while restoring the initial structure of the web page. It is possible to view downloaded websites without access to the network (by default loaded in the form of addresses for online viewing).

Initially, the WGET adapted to slow and unstable connection types, so its important feature is the ability to restore the interrupted file load (this is possible if the server supports the Range headers). Also, WGET can monitor the status of the servers (their availability) for long loading and updating (WGET checks the file headers and supports the latest version) of files and web pages.

The program also has a number of opportunities typical of so-called utilities, which explore various websites and servers for the presence of specific files on them.

Supported platforms

WGET Linux has become a base for this product. As mentioned above, it was originally a utility worked only on Linux, but quickly gained popularity and was adapted to other systems. For example, you can find the WGET version for Windows 7 and more modern OS, and there is assemblies with graphical interface. As for Linux, then there the program is often sewn into the distribution. Terminals In some Linux builds initially support WGET commands. Ubuntu, Fedora, OpenSUSE and other consumer distributions work with the program, which is called, out of the box. Next, consider the program in more detail.

WGET commands: list of basic parameters

The basic list of commands include:

  • -V - allows you to find out which version of the program is currently used;
  • -h - shows the most full list commands for working with the application;
  • -b - allows you to translate the program in the background mode;

List of message parameters

WGET commands allow you to fully control the displayed information. Here are their list:

  • -o logfile - creates a file log in which the entire application process is recorded;
  • -a logfile - allows you to supplement an existing file magazine instead of creating a new one;
  • -D - Displays the necessary developers and debugger information (helps solve problems and correct errors in the client's work);
  • -Q - disables messages display;
  • -V - this option is enabled by default, provides a detailed report on all processes occurring in the program (real-time);
  • -NV - significantly reduces the information received by the user in messages. Only the most necessary data and error alerts remain;
  • -i file (indicating the path to the file) - the command allows you to scan the file by delaying the link from it and download; You can use files of various formats, including HTML, but in the case of using this command, you should not specify the path to the file in the command prompt;
  • -f - provokes a program for reading a file as HTML; When using this command to work with local HTML Documentmay have an error to avoid it, it is enough to specify in the document .

List of download options

Basic teams to work with WGET help. Here are their basic set:

  • --Bind-address \u003d Address - This command allows you to transfer the BIND data to the Local ADDRESS. When using the TCP / IP connection, the command is required to accurately specify the IP address (if there are several).
  • -T Number (you need to specify the number) - allows the user to select the number of repetitions of the download. To avoid repetitions, specify the Number value equal to zero.
  • -Nc - This command allows you to return the previous status of the file being downloaded if the network fails, and the download was interrupted. If you do not use one of the commands for downloading (-n, -nc, -r), the WGET will create a copy of it and starts uploading it again. The -R parameter allows you to replace existing files.
  • -We - also contributes to the file transfer, in the event of a connection loss (and when WGET has stopped self-restored connection attempts). Just after reference to the file, specify this command. If the directory used exists a similar file, the WGET will hold its identification and automatically will decide whether to make it possible. The command can only be used on those servers, where there is support for Range headers.

  • --Progress - Allows you to select the type of display progress. You can specify -Progress \u003d BAR or PROGRESS \u003d DOT.
  • --spider - replaces the file download command to check its availability on the server;
  • -w (in seconds) - indicates the break time between downloads.
  • -Q (indicated in the number of kilobytes or megabytes) - allows you to clearly adjust the size of the downloadable data and limit excessive volumes. Nevertheless, it is worth noting that one file will be loaded independently of the quota exhibited (then you mean with one file this option will not work, whatever it is not heavy).

List of HTTP parameters

  • --Http-user \u003d and --http-passwd \u003d - allow you to specify the username and password for authorization on the web resource from which the files will be unloaded.
  • -C \u003d ON / OFF - allows you to enable or disable caching data on the server side.
  • --Cookies \u003d ON / OFF - the same as for Cache, only for cookies. Allows either stopping, or resume the exchange of cookies, the necessary server to keep visiting statistics. By default, the option is enabled, but cookies are not saved to the hard disk.
  • --Save-Cookies - allows you to save cookies files (Fresh data only).
  • --proxy-user and -proxy-passwd - allows you to specify a login and password for authorization on a proxy server (if such is activated).

List of FTP settings

Advanced options for working with WGET. FTP commands:

  • -NR - prohibits removing temporary files from the directory.Listing, which are spontaneously generated when working with FTP;
  • -g ON / OFF - allows you to use the Special Symbols utility in the FTP directories;
  • - PASSIVE-FTP - This command can be used if there is a firewall to activate the Passive FTP mode (when the user is responsible for setting the connection with the server).

WGET commands: Examples of use

The simplest and trivial example for working with the program is, of course, downloading files. To test the WGET commands, you can start with a simple, find a link to any file online and enter into the Console / Terminal: Wget * link indicating the exact and full path to the file *.

To continue downloading the file, use: wget -c * link indicating the exact and full path to the file *.

As for the download of web pages, everything is more complicated here, but the number of possibilities increases significantly. In order not to enter the parameters again each time, it was necessary to upload a website, you can create scripts or register all settings to file.wgetrc.

In order for the same page to be available for viewing without connecting to the Internet, you can use the WGET -R -10 -K command * link indicating the exact and full path to the file *.

The WGET format error The command is not found can mean any typical and incorrectly specified parameter.

We all sometimes swing files from the Internet. If you use programs with a graphical interface for this, then everything turns out to be extremely simple. However, when working at the Linux command line, it is somewhat more complicated. Especially - for those who are not familiar with suitable tools. One of these tools is an extremely powerful WGET utility, which is suitable for all types of downloads. We bring to your attention twelve examples, disassemble that, you can master the main features of Wget.

$ wget https://downloads.sourceforge.net/project/nagios/nagios-4.x/nagios-4.3.1/nagios-4.3.1.tar.gz?r\u003d&ts\u003d1489637334&use_mirror\u003dexcellmedia
After entering such a command, the Nagios Core is started. During this process, you can see the download data, for example, information about which volume is already loaded, current speed, and how long it remains until the end of the download.

2. Download file and save it with a new name.

If we want to save the downloaded file under the name, different from its original name, we will use the WGET command with the -o parameter:

$ Wget -o Nagios_Latest https://downloads.sourceforge.net/project/nagios/nagios-4.x/nagios-4.3.1/nagios-4.3.1.tar.gz?r\u003d&ts\u003d1489637334&use_mirror\u003dexcellmedia
With this approach, the downloaded file will be saved under the name Nagios_Latest.

3. File download speed limit

If necessary, the speed of downloading files using WGET can be limited. As a result, this operation will not occupy the entire available data transfer channel and will not affect other processes associated with the network. This can be done using the --Limit-Rate parameter and specifying a speed limit expressed in bytes (as a conventional number), kilobytes (adding after the number k) or megabytes (M) per second:

$ Wget --Limit-rate \u003d 500K https://downloads.sourceforge.net/project/nagios/Nagios-4.x/nagios-4.3.1/Nagios-4.3.1.tar.gz?r\u003d&ts\u003d1489637334&use_mirror \u003d Excellmedia
Here the load speed limit is set equal to 500 kb / s.

4. Completing the interrupted load

If during the download files this operation was interrupted, you can resume the download using the -c command parameter WGET:

$ wget -c https://downloads.sourceforge.net/project/nagios/nagios-4.x/nagios-4.3.1/nagios-4.3.1.tar.gz?r\u003d&ts\u003d1489637334&use_mirror\u003dexcellmedia
If this option is not used, then the loading file will start up first.

If you download a huge file and want to perform this operation in the background, this can be done using the -b parameter:

$ wget -b https://downloads.sourceforge.net/project/nagios/nagios-4.x/nagios-4.3.1/nagios-4.3.1.tar.gz?r\u003d&ts\u003d1489637334&use_mirror\u003dexcellmedia

If there is a list of files of files that need to be downloaded, but you do not want to manually start downloading these files, you can use the -i parameter. However, before starting the download, you need to create a file containing all addresses. For example, you can do this with such a team:

$ VI url.txt
You need to place the addresses in this file - one at each line. Next, it remains only to start WGET, passing this utility that the download file has just created:

$ wget -i url.txt
The execution of this command will lead to one-time download of all files from the list.

7. Increase the total number of file download attempts

In order to configure the number of repetition of the file loading, you can use the --Tries parameter:

Wget --tries \u003d 100 https://downloads.sourceforge.net/project/nagios/Nagios-4.x/nagios-4.3.1/nagios-4.3.1.tar.gz?r\u003d&ts\u003d1489637334&use_mirror\u003dexcellmedia

The file download command from an anonymous FTP server with WGET looks like this:

$ wget ftp-url
If the user name and password are required to access the file, the command will take this kind:

$ wget --ftp-user \u003d dan --ftp-password \u003d ********* FTP-URL

9. Creating a local copy of the website

If you want to download the contents of the whole website, you can do this by using the --Mirror parameter:

$ Wget --Mirror -p --Convert-Links -p / Home / Dan xyz.com
Pay attention to the Advanced Command Line Settings:

  • -p: All files needed to be downloaded for the correct display of HTML pages.
  • --Convert-Links: Links in documents will be converted for local site viewing purposes.
  • -P / Home / Dan: Materials will be saved to the / Home / Dan folder.

10. Loading from the site only files of a specific type

In order to download only a specific type files from the site, you can use the -R parameters:

$ wget -R -a.txt WebSite_URL

11. Skip files of a certain type

If you want to copy a whole website, but you do not need files of a specific type, you can disable their download using the --reject parameter:

$ wget --reject \u003d PNG WebSite_URL

12. Download using your own.log file

In order to download the file and use your own.Log file, use the -o parameter and specify the name of the log file:

$ wget -o wgetfile.log https://downloads.sourceforge.net/project/nagios/nagios-4.x/nagios-4.3.1/nagios-4.3.1.tar.gz?r\u003d&ts\u003d1489637334&use_mirror\u003dexcellmedia

RESULTS

WGET is pretty simple to use, but very useful utility Linux. And, in fact, what we told - only a small part of what she can. We hope this review will help those who were not familiar with WGET, evaluate this program, and possibly include it in your everyday arsenal of command line tools.

Dear readers! Do you use Linux command line tools to download files? If so, we ask them to tell.

GNU WGET is a free non-interactive console program for downloading files over the network. Supports HTTP, FTP and HTTPS protocols, and also supports work via HTTP proxy server. The program is enabled in almost all GNU / Linux distributions.

GNU WGET is a non-interactive program. This means that after its launch, the user can affect its operation only with the help of process management tools for the operating system itself. As a rule, the Ctrl + C key combinations are used for the need to interrupt the program and Ctrl + Z to place the current task in the background.

Modern browsers usually have the function of downloading files, however, since the browser is designed for an interactive mode of operation, then downloading a large number of files manually can be tedious. Browsers, as a rule, do not provide means to automate such tasks. GNU WGET, for example, supports the URL boot specified in the file. Thus, you can make a list of files, and at any convenient time to download them using GNU WGET.

The command line interface allows you to manage the GNU WGET from other programs and scripts, which is used when automating the file download (regular updates, server availability monitoring, etc.).

GNU WGET allows you to download any files in the World Wide Web (including (x) HTML Pages) using HTTP and HTTPS protocols, as well as files and directory lists using the FTP protocol.

Files can be downloaded recursively by references in HTML pages, both from one site from a certain depth of following links and from several. In addition, when loading FTP files You can download "on the mask" name (that is, you can specify using "*" file group).

Also, GNU WGET supports the file fraction in the event of a compound break.

Introduction to GNU Wget

GNU Wget IS A Free Software Package for Retrieving Files using HTTP, HTTPS, FTP and FTPS THE MOST WIDELY-USED Internet Protocols. IT IS A NON-INTERACTIVE COMMANDLINE TOOL, SO IT MAY EASILY BE CALLED FROM SCRIPTS, JOBS, TERMINALS WITHOUT X-Windows Support, etc.

GNU Wget Have Many Features to Make Retrieving Large Files Or Mirroring Entire Web or FTP Sites Easy, Including:

  • Can Resume ABORTED Downloads, using and
  • CAN USE FILENAME WILD CARDS AND RECURSIVELY MIRROR DIRECTORIES
  • NLS-Based Message Files For Many Different Languages
  • Optionally Converts Absolute Links In Downloaded Documents to Relative, So That Downloaded Documents May Link to Each Other Locally
  • Runs On Most Unix-Like Operating Systems AS Well As Microsoft Windows
  • Supports http proxies
  • Supports http cookies
  • Supports Persistent HTTP Connections
  • Unattended / Background Operation
  • Use Local File TimeStamps to Determine Whether Documents Need To Be Re-Downloaded WHEN MIRRORING
  • GNU Wget Is Distributed under the Gnu General Public License.

The Source Code for GNU Wget Can Be Found On Here Or ON OUR Mirrors.
For More Download Options, See The FAQ.

Documentation

GNU WGET Documentation Can Be Found Here.
For MANUALS OF OTHER GNU PACKAGES, PLEASE SEE www.gnu.org.

ADDITIONAL INFORMATION.

Currently GNU Wget2 is Being Developed.
Please Help US If You Can With Testing, Docs, Organization, Development, ... See You at Wget2 Collaboration Site

Maintainer.

GNU Wget IS Currently Being Maintained by Tim Rühsen, Darshit Shah and Giuseppe Scrivano.
The Original Author of Gnu Wget IS Hrvoje Nikšić.
Please. do. not. Directly Contact Either Bug Reports, Or Requests for Help With Wget: That Is What The Mailing List Is For; Please Use It Instead.

Donators.

We Thak The Typo3 & Magento Partner Aoe Media for Donating To the Wget Project.

wGET - lady downloads
(Download files and sites from the Internet in GNU / Linux)

About love wget so many songs spito
I will sing, you will sing, one more ...

Romance from k / f "Three more"

A task: I want Teleport Pro and FlashGet for Linux, download sites and files.

Decision: Discover for yourself wGET. - It's all in one bottle and even more.

It is often searched for a lot of graphic programs with several buttons - but there is one utility that can replace them and providing the possibilities that are no longer anywhere.

it wGET. And all that is based on it. The development of this utility requires some effort and at least a quick reading of the manual, but in return you receive a powerful utility that will save your time and traffic.

It is worth saying that mAN WGET. At first glance, makes a crushing impression, but then you understand that not all options are needed in everyday use. Below are the most frequently used options.

Using a la "FlashGet for Linux"

You just need to download a file and / or continue downloading, which broke off. It is very and very easy to do, writing several lines in the terminal.

Just download the WGET file:

$ wget ftp://vasya.pupkin.com/film.avi.

Another option: part of the file downloaded, the download was cut off.

To continue the file download, write:

$ wget -c ftp://vasya.pupkin.com/film.avi or $ wget -continue ftp://vasya.pupkin.com/film.avi

Instead, it is possible to write -Continue, as each key has a short and long shape: the long keys are easier to remember, but to write longer. You can easily mix various writing forms.

To pump out files from the list containing direct links:

$ wget -i pupkinlist or $ wget -input-file \u003d pupkinlist

It is only indicated here filewhich contains links. The file can also be an HTML page in which there are links.

Windows. Installing WGET in Windows XP / 7/8 / 8.1 / 10

They will be sold out the above command.

Using a la "Teleport Pro for Linux »

When downloading websites opportunities more, and therefore more keys are required. Again, it's not necessary to remember them, you can make a script (and better somewhat - for different cases) and cause them.

Mirroring sites on the local car:

$ wget -m http://www.vasyapupkin.com/

Copying the site for local viewing:

$ WGET -R -L0 -NP -K http://www.vasyapupkin.com/

This will include a recursive unloading (key -R, -Recursive), that is, not only files from the main page, but also all the other on which the links lead (key -L0 infinite reference nesting).

Link names will be converted to local for convenience of viewing (key -K). Also, using the -np key (no-parrent), you can prohibit WGET climb above the initial address during recursive download, that is, if you copy http://home.vasyapupkin.com/ This links from the main site http: // www. vasyapupkin.com/ will not be copied.

Several useful keys

Enabling and excluding files when downloading:

A Acclist or -Accept Acclist -R Rejlist or -reject Rejlist

Specifies the files shared by commas templates that should be downloaded (acclist) or ignore (Rejlist).

K or -Convert-Links

Turns absolute links (type http: // www ...) to relative (type File /// Home / Vasya / www / index.html) for the convenience of local viewing. To view the downloaded site in the browser locally, open the index.html file in the browser and root along the links just as if you were connected to the Internet.

H or -Span-hosts

Allows WGET download data from any address to which there is a link in the requested document.

P or -Page-Requisites

Loads all the files that are needed to display HTML pages. For example: drawings, sound and cascade styles. After the download is complete, it converts links to a document for viewing offline. This concerns not only visible references to other documents, and links to all external local files.

siteget - a simple script for downloading sites

If you are tired of all these wisdom with keys wGET. And you just need to download the site, then siteget. You will definitely like it. This is a fairly simple script with the necessary parameters for downloading the site.

You can download the script from here. After downloading to the computer, it makes sense to copy it in a superuser mode into one of the directories prescribed in the $ PATH variable: operating system Automatically scans them in search of executable files. For example, to the directory / User / Local / Bin:

# CP (Directory where Script is located) / SieceT / User / Local / Bin / Siecet

The script also needs to assign an executable file attribute:

# Chmod A + X / USR / Local / Bin / Sieget

Using siteget. Very simple:

$ Sitecome http://www.vasyapupkin.com/

Department of Biblical MDA, 2008.
Last update:
Address on the Internet: http://www.bible-mda.ru/soft/wget/wget.html

Simply download file WGET:

wget ftp://vasya.pupkin.com/film.avi.

For continue broken injection We write:

wget -c ftp://vasya.pupkin.com/film.avi.

wget -Continue ftp://vasya.pupkin.com/film.avi.

As in other programs, the keys have a short and long form, and instead you can write. Long keys are easier to remember, but to write longer. You can easily mix various writing forms.

To roll out files from the listcontaining direct links:

wget -i pupkinlist.txt

wget -Input-File \u003d pupkinlist.txt

Here only the file contains links. The file may also be -prack, in which there are links. They will be sold out the above command.

Use a la "Teleport Pro for Linux".

When downloading websites opportunities more, and therefore more keys are required.

Again, it's not necessary to remember them, you can make a script (and better somewhat - for different cases) and cause them.

So, if there is a website, and I would like to have its local copy on the computer, so that, turning off from the network, it was possible that it was not in a hurry to read it.

Mirroring sites on the local car:

wget -m http://www.vasyapupkin.com/

equivalently, these options are described below.

Copying the site for local viewing (with replacing Internet links to local addresses downloaded pages):

wget -R -L0 -K http://www.vasyapupkin.com/

This will include a recursive unloading (key -R, -Recursive),

In Wget there is a large number of useful options - more than Teleport "And flags. Being wrapped in a script, for example, and laid for a prominent place (specified in), we have ease of use and wealth of settings.

Do not rise above the initial address during recursive download.

Enable recursive viewing directories and subdirectories on a remote server.

Determine the maximum depth of recursion equal to depth when viewing directories on a remote server. Default depth \u003d 5.

Do not go to the parent directory while searching for files. This is a very useful property because it ensures that only those files that are below a certain hierarchy will be copied.

A list of file names separated by commas, which should (accept) or should not (REJECT) download. It is allowed to set the file names on the mask.

WGET for Windows. Installation and use.

Turn the absolute links in the document to relative links. Only those references that indicate actually downloaded pages will be converted; The rest will not be transformed. Note that only at the end of the work of WGET will be able to find out which pages were really loaded. Consequently, only at the end of the WGET work will be performed final transformation.

Specify the username and password inserver.

Allows you to visit any servers to which there is a link.

Download all files that are needed to display pages. For example: drawings, sound, cascade styles (). By default, such files are not loaded. Parameters and specified together can help, but because WGET does not distinguish external and internal documents, then there is no guarantee that all required will be loaded.

Links

Information about the program

WGET is a console utility for downloading files using HTTP, FTP protocols and proxy connections.

wget for Windows.

The program allows you to copy simple HTML. Sites with saving directories. Allows you to find and process the Robots.txt data, it is able to convert links for the subsequent offline access. Contains a tool to compare file headers to download new versions and create mirror copies.

The application supports the resumption of work when an error occurs or breaking the coupling, a recursive jump indicating the name mask and the depth of the exercise links. Knows how to log on FTP server.connecting in the background to the protected HTTPS sites.

The program is able to conduct a detailed log, is the optimal means for creating mirrors of remote storage or web resources, streaming data from FTP servers.

Download free full Russian version of WGET from the official site without registration and SMS.

System requirements

  • Supported OS: Windows XP, 7, 10, 8, Vista, 8.1
  • Bitness: 32 Bit, x86, 64 bit

WGET - Console bootloader for downloading sites and its use in Windows

wGET - Console utility for downloading files / sites, can scroll files recursively, following the links automatically.

Team Description
wget http://example.com/file.zip. download file.zip file to the current directory
wget -p / Path / To / Save http://example.com/file.zip download file.zip file to directory / Path / To / Save
wget -c http://example.com/file.zip. switch file.zip file in cases of cliff
wget -o Arch.zip http://example.com/file.zip download file.zip file and saving under the name Arch.zip
wget -i Files.txt download files from the list in Files.txt
wget -tries \u003d 10 http://example.com/file.zip number of download attempts
wget -Q5m -i http://example.com/ quota on maximum size Downloaded files, the quota is valid only with recursive download (-R)
wget -Save-cookies cookies.txt -post-data 'username \u003d PROFT & PASSWORD \u003d 1' http://example.com/auth.php identification on the server with the preservation of Cook for subsequent access
wget -user-Agent \u003d »Mozilla / 5.0 (x11; linux x86_64) AppleWebKit / 536.5 (KHTML, LIKE GECKO) Chrome / 19.0.1084.9 Safari / 536.5" http://example.com/ note User Agent
echo 'Wget http://example.com/file.zip' | at 12:00 download http://example.com/file.zip at 12:00 to the current directory
wget ftp://example.com/dir/*.zip.
wget http://example.com/dir/file (1..10).zip. download all files by template
wget -s http://example.com/ output HTTP server headers and responses FTP servers
wget -spider -i URLS.Txt check links in the file for availability
wGET -B http://example.com/file.zip. downloading a file in the background, log is written in wget.log, wget.log.1, etc.
export http_proxy \u003d http: //proxy.com: 3128 /; Wget http://example.com/file.zip download file * file.zip * through proxy
wGET -M -W 2 http://example.com/ mirroring of the site with the preservation of absolute links and waiting 2 seconds between requests
wget -Limit-Rate \u003d 200k http://example.com/file.zip restricting the speed of download
wget -R BMP http://example.com/ don't download bmp files
wget -a PNG, jpg http://example.com/ download only pNG files and jpg.

Example of use for downloading documentation Django.:

  • - We walk on the links (recursive download)
  • - convert links to local species
  • - downloading resources required to display an HTML page (styles, pictures, etc.)
  • - the depth of download, 0 - infinite nesting links
  • - Do not overwrite existing files
  • - Do not rise above the initial address during recursive download

Often used wGET. Parameters can be taken out in ~ / .wgetrc.

cURL - Console Data Utility Using URL Syntax, Supports DICT, File, FTP, FTPS, Gopher, HTTP, HTTPS, IMAP, IMAPS, LDAP, LDAPS, POP3, POP3S, RTMP, RTSP, SCP, SFTP, SMTP supported SMTPS, TELNET and TFTP.

Team Description
cURL http://proft.me. we get the main page content
curl -o index.html http://proft.me get the main page content to the index.html file
curl -l http://example.com. when you receive the contents of the page to follow the redirects (if any)
cURL -U UserName: Password http://example.com/login/ getting a page hidden for Basic HTTP Authentication
curl -X Proxy.com:3128 http://proft.me getting a Page Using Proxy
cURL -I PROFT.ME. we get http headers from the site
cURL -H 'HOST: Google.ru' http://proft.me create a domain when accessing the server (transferring your header)
cURL -REQUEST POST "http://example.com/form/" -Data "Field1 \u003d Value1 & Field2 \u003d Value2"
cURL -X POST "http://example.com/form/" -Data "Field1 \u003d Value1 & Field2 \u003d Value2" data transfer Post-query
cURL -X POST -H "CONTENT-TYPE: Application / JSON" -D '"Title": »Commando», »Year»: »1985"' 'http://example.com/api/movies/ data transfer POST request, JSON data
cURL -REQUEST PUT "http://example.com/api/movie/1/" -Data "title \u003d djangoonchained" data Transfer Put-Request
curl -f. [Email Protected] -F submit \u003d ok http://example.com/upload/ download file.zip file shape (Multipart / Form-Data)
cURL -U UserName: password -o ftp://example.com/file.zip download file with ftp
cURL -U UserName: Password -t file.zip ftp://example.com/ download FTP file
cURL -COOKIE "LOGIN \u003d PROFT" http://example.com/login/ install cookies
cURL -COOKIE-JAR COOKIES.TXT http://example.com saving cookies to file
curl -Cookie cookies.txt http://example.com/login/ use of preserved cookies

Additional fiction